Showing posts with label guys. Show all posts
Showing posts with label guys. Show all posts

Wednesday, March 28, 2012

poor index performance

Hey guys,

Having some trouble with indexes on sql server 2005. I'll explain it with a simplified example.
I have a customers table, and a sp to list customers :

create table Customers(
CusID int not null,
Name varchar(50) null,
Surname varchar(50) null,
CusNo int not null,
Deleted bit not null
)

create proc spCusLs (
@.CusID int = null,
@.Name varchar(50) = null,
@.Surname varchar(50) = null,
@.CusNo int = null
)
as

select
CusID,
Name,
Surname,
CusNo
from
Customers
where
Deleted = 0
and CusID <> 1000
and (@.CusID is null or CusID = @.CusID)
and (@.CusNo is null or CusNo = @.CusNo)
and (@.Name is null or Name like @.Name)
and (@.Surname is null or Surname like @.Surname)
order by
Name,
Surname

create nonclustered index ix_customers_name on customers ([name] asc)
with (sort_in_tempdb = off, drop_existing = off, ignore_dup_key = off, online = off) on primary

create nonclustered index ix_customers_surname on customers (surname asc)
with (sort_in_tempdb = off, drop_existing = off, ignore_dup_key = off, online = off) on primary

create nonclustered index ix_customers_cusno on customers (cusno asc)
with (sort_in_tempdb = off, drop_existing = off, ignore_dup_key = off, online = off) on primary

I've recently noticed that some tables, including 'Customers' don't have indexes except primary keys. And I have added indexes to "name", "surname" and "cusno" columns. This has dropped the number of IO reads. But the strange thing is; one time it works with name / surname searches like ('joh%' '%') but when CusNo is included, it does a full scan. And vice versa when the SP is recompiled using 'alter', works ok with CusNo, but not with name/surname. Recompile it, and it's reversed again. When run as a single query, the execution plan looks different.

What's happening? Perhaps something to do with statistics? This doesn't have a big payload on the server, but there are some other procs suffering from this on heavy queries, making server performance worse than before...

You will get the best performance if you can create a "covering" index for this query, which is a non-clustered index that includes all of the columns needed to satisfy or "cover" the query.

In this case, I would try a unique, non-clustered index on Deleted, CusID, CusNo, Name, and SurName (all of these in a single NC index). You may have to play around with the order of the columns in this index (based on their selectivity) to get the best results.

Also, the fact that you are using OR and LIKE in your WHERE clause will cause performance issues. I would consider splitting this into two SP's instead of trying to use an "all-purpose" SP for this.

sql

Wednesday, March 21, 2012

Point in Time query >:[

Hey guys and gals,

I'm having a real problem with this query at the moment...
Basically I have to produce a query which will tell me the total number of people employed by the company at any given date and the total salary for all these people.

We have a people table and a career table.
People(unique_identifier, known_as_and_surname, start_date, termination_date ...)
Career(unique_identifier, parent_identifier, career_date, basic_pay ...)
Relationship people.unique_identifier = career.parent_identifier

Employees can be identified like so

SELECT *
FROM people
WHERE start_date <= DateSelected
AND (termination_date > DateSelected
OR termination_date IS NULL)

Passing the selected date to the query is no trouble at all I am just having problems with the point in time side of this.

All and any help is greatly appreciated :)
~George

P.S. SQL Server 2000 ;)...I am just having problems with the point in time side of this.could you elaborate on this a bit please?

because your query looks fine, all you need is an INNER JOIN as well as a GROUP BY and some aggregate expressions|||george ... is People to Career a 1:1 or 1:many relationship? What if a personn's salary changes during the date range in question?

If 1:1, then a count of unique identifiers and sum of the salary from the Career table using a join on a filter from the People table using the date criteria would do the job.

If however, you can have a salary change during the data range, and you have two rows in the Career table, you will need someone to define the business rules for that situation.|||could you elaborate on this a bit please?

I need to find out if they were an employee at any given date...
There is no career_date_from or to fields, just a single career_date - which is part of the problem!

Tom, One person can have many career history lines. and the problem is - how do I make this a range?

Here's some samlpe data that may help

unique_id name start_date termination_date
00001 George V 01/11/2006 NULL
00002 Tom 53 01/06/2004 01/06/2007
00003 Rudy 937 07/07/2007 NULL

unique_id parent_id career_date basic_pay
1 00001 01/11/2006 150
2 00001 01/12/2006 165
3 00002 01/06/2004 155
4 00003 07/07/2007 160
5 00003 09/07/2007 170

If I entered 02/11/2006 as my criteria I'd want to return the sum of the following lines

unique_id parent_id career_date basic_pay
1 00001 01/11/2006 150
3 00002 01/06/2004 155

Which gives us
2 employees : £305|||Do you just want the "last" (by career_date) record in Career where the career_date is less than or equal to the DateSelected?|||I think that's it!
I believe that makes it look something like this:

DECLARE @.SelectedDate datetime
SET @.SelectedDate = '2006-11-02'

SELECT Count(*)
,Sum(c.basic_pay)
FROM people e
LEFT JOIN career c
ON c.parent_identifier = e.unique_identifier
AND c.career_date = (
SELECT Max(career_date)
FROM career
WHERE parent_identifier = c.parent_identifier
AND career_date <= @.SelectedDate
)

That's what I couldn't get my head around :)|||george ... reference uniqueid '00002' ... you can't fire me ... I QUIT ;)|||george, your sample data was the key to understanding the data relationship (which was not at all apparent from post #1)

just another example of why we ask posters to show sample data :cool:

p.s. those unique_ids are awful!|||They are aweful, but you know what...
It allowed the original developers to make an inbuilt query designers that fools can use - which helps me a little. The only other benefit is that you know the relationships between almsot everything simply by logic.

But yes, it was like this when I got it :p

Can one of you kindly check the following code over once? It's my "final" result

DECLARE @.SelectedDate datetime
SET @.SelectedDate = '2005-11-01'

SELECT Count(*)
,Sum(c.basic_pay)
FROM people e
LEFT JOIN career c
ON c.parent_identifier = e.unique_identifier
AND c.career_date = (
SELECT Max(career_date)
FROM career
WHERE parent_identifier = c.parent_identifier
AND career_date <= @.SelectedDate
)
WHERE (e.termination_date > @.SelectedDate
OR e.termination_date IS NULL)
AND e.start_date <= @.SelectedDate

Oh and Tom, you were never fired... You just didn't turn up ;)

And Rudy; yes it occured to me that I never mentioned that it was 1:M...
It's Monday, I'm frazzled already!
I came in this morning and my monitor was covered in sticky notes because of missed calls etc. So lame.
Not a good start to the week.

Finally - thank you all :)|||Once again Poots' incisive logic cuts to the very core of the problem :cool:

Ok - this:
SET @.SelectedDate = '2005-11-01'
is not guarenteed to work in all system set ups. Better is:
SET @.SelectedDate = '20051101'
Also - is there a unique constriant on the composite key parent_identifier, career_date (assuming a person cannot have two career records in a day)? If not there could be two records for a person on a given day -> errors in the count and sum.|||SET @.SelectedDate = '2005-11-01'

This query will be translated into a 3rd party program that will only run on SS 2000 with it's own run time expression builder - so this was purely for testing purposes ;)
You can have two career records on one day... which is something I had not thought about. Would an order by clause sort this out (if say, I ordered it by date_entered)?|||Damnit - the dupes are causing me a problem.
There are around 10 people who are being counted twice!

How can I eliminate these?|||Extend the same logic again. You wanted the max(career_date). You now want the max(date_entered) for the max(career_date)...|||Yeah... Having trouble with that.

AND c.career_date =
(
SELECT Max(x.career_date)
FROM career x
WHERE x.parent_identifier = c.parent_identifier
AND x.career_date <= @.SelectedDate
AND x.created_by_user =
(
SELECT Max(created_by_user)
FROM career
WHERE parent_identifier = c.parent_identifier
AND career_date <= @.SelectedDate
)

Is not what I want... I'm sorry; lack of sleep + stress =
...can't even think of a word suitable to complete that sentence :( *sigh*|||Untested but worrabout:
... AND c.career_date = (
SELECT TOP 1 career_date
FROM MySchema.career
WHERE parent_identifier = c.parent_identifier
AND career_date <= @.SelectedDate
ORDER BY career_date DESC, created_by_user DESC
)|||SQL Server 2000 - no TOP *sigh*|||SQL Server 2000 - no TOP *sigh*Yeah - you mentioned that before. It is in SQL 2k.

What error do you get again?|||You now want the max(date_entered) for the max(career_date)...hmmm, sounds mysteriously like minimum price on earliest date (http://www.dbforums.com/showthread.php?t=1618384)

:cool:|||Sounds like it - but I can't use local views - courtesy of SS2K :o

However, I think I may have cracked it!
It's not pretty, but (I'm fairly sure :p) it works!

SELECT Count(*)
,Sum(c.basic_pay)
FROM people e
LEFT JOIN career c
ON c.parent_identifier = e.unique_identifier
AND c.career_date = (
SELECT max(c2.career_date)
FROM career c2
WHERE c2.parent_identifier = c.parent_identifier
AND c2.career_date <= @.SelectedDate
)
AND c.datetime_created =(
SELECT max(c3.datetime_created)
FROM career c3
WHERE c3.parent_identifier = c.parent_identifier
AND c3.career_date = c.career_date
)
AND (e.termination_date > @.SelectedDate
OR e.termination_date IS NULL)
AND e.start_date <= @.SelectedDate

What you think? :)|||Looks fine. Shame it takes three scans of the careers table :confused:

I think you should post the lack of TOP as a thread - it ain't right I tell ya it ain't right. Check it out in BoL - should be there.|||Yep - it's not exactly the fastest query in the west, but on this occasion I'm going to let it slide - because I actually can't come up with a better solution!

Why should the lack of TOP be a thread?
I suppose it is odd that it recognises TOP as a keyword (highlights it blue)

SELECT TOP 1 FROM people ORDER BY birth_date DESC
-----
Server: Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'FROM'.|||Why should the lack of TOP be a thread?
I suppose it is odd that it recognises TOP as a keyword (highlights it blue)

SELECT TOP 1 FROM people ORDER BY birth_date DESC
-----
Server: Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'FROM'.
Duhhhhhh! SELECT TOP 1... what?

SELECT TOP 1 *, myfield, 'George is a plonker' AS plonky_george
FROM people
ORDER BY birth_date DESC
-----
Server: Msg 156, Level 15, State 1, Line 1
Incorrect syntax near the keyword 'FROM'.|||More interestingly

AND c.datetime_created =(
SELECT max(c3.datetime_created)
FROM career c3
WHERE c3.parent_identifier = c.parent_identifier
AND c3.career_date = c.career_date
ORDER BY c3.career_date DESC
)

Gives me

Server: Msg 1033, Level 15, State 1, Line 21
The ORDER BY clause is invalid in views, inline functions, derived tables, and subqueries, unless TOP is also specified.

It's just toying with me!|||That is invalid and also the order by is superfluous there anyhoo.|||Apologies

SELECT TOP 1 birth_date FROM people ORDER BY birth_date
----
Server: Msg 170, Level 15, State 1, Line 1
Line 1: Incorrect syntax near '1'.

EDIT: How many times will I make the same mistake?|||Apologies

SELECT TOP birth_date FROM people ORDER BY birth_date
----
Server: Msg 207, Level 16, State 3, Line 1
Invalid column name 'TOP'.
Duhhhhhhhhhhh! ;)
SELECT TOP how many?|||this time you forgot the top how many

come on, george, slow down and do some desk checking (an age-old debugging technique where you actually read what you just wrote to see if it makes sense)

:)|||See above (edit) :D

My edit beat your posts ;)

And yes, sorry :(

And another smiley for luck :cool:

It's been a long hectic day :o|||Ok - so can we confirm that the below query and error go together?

SELECT TOP 1 birth_date FROM people ORDER BY birth_date
----
Server: Msg 170, Level 15, State 1, Line 1
Line 1: Incorrect syntax near '1'.

If so George - start a thread. This is some sort of error.|||Yes, that is correct.
Damnit.
*goes off to start a new thread* :rolleyes:

Thank you for your patience - and I'm glad you had fun when I misposted ;)

SELECT TOP 1 *, myfield, 'George is a plonker' AS plonky_george
FROM people
ORDER BY birth_date DESC|||I'm glad you had fun when I misposted ;)When you are as easily amused as me every day is a riot ;)

Friday, March 9, 2012

pls help

Hi guys,
i appreciate the speed at which the response come and the quantity as
well.
i want an identity column with repetitions allowed after a certain
condition is reached.
actually, i want to generate invoice number automatically. it will be
an integer data type.
but the seed should initialize every time when the financial year is
changed.
for example, you have to display invoice number as 1/2005 upto
1200/2005
and we should have both of these fields different viz year and invoice
number. then we will contecate them using a slash as shown.
on the next financial year it will be changed as 1/2006 upto 1200/2006.
i want a stored procedure for this that should return an identity.
thanks and regards
Suresh BeniwalSuresh
for year you can simply use YEAR(GETDATE()) AS DEFAULT.
for invoice number, I think one should not use identity. use int and
increment in the stored procedure/client
--
Regards
R.D
--Knowledge gets doubled when shared
"SureshBeniwal" wrote:

> Hi guys,
> i appreciate the speed at which the response come and the quantity as
> well.
> i want an identity column with repetitions allowed after a certain
> condition is reached.
> actually, i want to generate invoice number automatically. it will be
> an integer data type.
> but the seed should initialize every time when the financial year is
> changed.
> for example, you have to display invoice number as 1/2005 upto
> 1200/2005
> and we should have both of these fields different viz year and invoice
> number. then we will contecate them using a slash as shown.
> on the next financial year it will be changed as 1/2006 upto 1200/2006.
> i want a stored procedure for this that should return an identity.
> thanks and regards
> Suresh Beniwal
>|||and dont be misled by the word reseed. It wont set 1 again but the one more
than max number.So it is not possible to reset back to one when new year
starts.
DBCC CHECKIDENT ('table_name') or DBCC CHECKIDENT ('table_name', RESEED)
--
Regards
R.D
--Knowledge gets doubled when shared
"SureshBeniwal" wrote:

> Hi guys,
> i appreciate the speed at which the response come and the quantity as
> well.
> i want an identity column with repetitions allowed after a certain
> condition is reached.
> actually, i want to generate invoice number automatically. it will be
> an integer data type.
> but the seed should initialize every time when the financial year is
> changed.
> for example, you have to display invoice number as 1/2005 upto
> 1200/2005
> and we should have both of these fields different viz year and invoice
> number. then we will contecate them using a slash as shown.
> on the next financial year it will be changed as 1/2006 upto 1200/2006.
> i want a stored procedure for this that should return an identity.
> thanks and regards
> Suresh Beniwal
>|||Thanks a lot, R.D.
that should help
Regards
Suresh Beniwal|||>> want to generate invoice number automatically. it will be an integer dat
a type. but the seed should initialize every time when the financial year is
changed. <<
You did check to see that it follows the conventions of your accountng
department and accounting software first? I doubt it.
Does your accounting department know that you are planning to destroy
the audit trail? IDENTITY has gaps, no check digits, etc. You need to
stop and learn how to design codes. If you want to follow this pattern
put the year first, then a sequence number, then a check digit; make
sure it is fixed length. You do not want to give away information, you
can use an additive congruential generator to get values in
pseudo-random order.

Please suggest an index

Could you guys give me some feedback on the below query and suggest the
best possible query assuming this is the only query that will be ran
against these tables?
The destination has 200 million records and the staging table has 30
million records.
col1_int has 1500 unique values
col2_int has 50000 unique values
col3_char_55 has 50000 unique values
INSERT INTO myTable
SELECT a.*
--SELECT count(*)
FROM myTable_stage a with(nolock)
LEFT OUTER JOIN myTable b with(nolock) ON b.col1_int =
a.col1_int
AND b.col2_int = a.
col2_int
AND b.col3_char_55 =
a.col3_char_55
WHERE b.col1_int IS NULLCorrection... I meant the best possible index or indexes not "best
possible query".|||Are you actually plannign on returning every single row every time you run
this view?
I can't imagine this is the case, based on your 200 million * 30 million
rows.
The fastest way to get back every row would probably be to put an index on
the 200 million row table on collumns (col3_char_55, col2_int, col1_int),
although havign the same collumns indexed on the table with 30 million rows
may work as well.
Returning this many rows will take ages, just to move the data over the
network, no matter how how fast the SQL Server engine performs the query.
What are you actually trying to do?
"Dave" <daveg.01@.gmail.com> wrote in message
news:1140103779.299793.117660@.g43g2000cwa.googlegroups.com...
> Could you guys give me some feedback on the below query and suggest the
> best possible query assuming this is the only query that will be ran
> against these tables?
> The destination has 200 million records and the staging table has 30
> million records.
>
> col1_int has 1500 unique values
> col2_int has 50000 unique values
> col3_char_55 has 50000 unique values
>
> INSERT INTO myTable
> SELECT a.*
> --SELECT count(*)
> FROM myTable_stage a with(nolock)
> LEFT OUTER JOIN myTable b with(nolock) ON b.col1_int =
> a.col1_int
> AND b.col2_int = a.
> col2_int
> AND b.col3_char_55 =
> a.col3_char_55
> WHERE b.col1_int IS NULL
>|||The above query will not return every row. Only the rows that are not
already in myTable.
The query inserts the new records from the staging table into the base
table.
Your index suggestion is what I was looking for. I was thinking
something like a clustered index on (col2_int, col3_char_55, col1_int)
would be best.|||Yes, I should have noticed the outer join and the b.col1_int IS NULL, and
realized what you were doing. I was sloppy and just looked at the join,
ignoring the rest of it.
Since you are doing the outer join, I believe the index would need to on the
destination table, and I think the columns (col3_char_55, col2_int,
col1_int) would work best.
A clustered index on your destination table may actually add overhead that
you don't want. You'll have to get someone elses opinion on whether a
clustered index is best for a table of this size, although more information
is probably needed to determine that. A regular index may work as well or
better.
You might also try using a not exists instead of the outer join just to see
the difference in performance.
"Dave" <daveg.01@.gmail.com> wrote in message
news:1140108936.361531.64340@.f14g2000cwb.googlegroups.com...
> The above query will not return every row. Only the rows that are not
> already in myTable.
> The query inserts the new records from the staging table into the base
> table.
> Your index suggestion is what I was looking for. I was thinking
> something like a clustered index on (col2_int, col3_char_55, col1_int)
> would be best.
>

Monday, February 20, 2012

Please help!! Urgent

Hi guys,

we have a database here and something happened which causes a database block. We tried to run the 'sp_who' 'active' command to see the spid which locked the database, and we found out that some transaction is blocking another transaction. The following is the sample data results from the sp_who 'active'

spid ecid status loginame hostname blk dbname cmd

52 0 sleeping HOSTING\SQLMonitor BLUE2 185 tempdb INSERT
53 0 sleeping sa 10.10.10.106 185 mfgq_live SELECT
56 0 sleeping sa 10.10.10.106 175 mfgq_live UPDATE
57 0 sleeping sa 10.10.10.143 185 mfgq_live SELECT

We killed all spid which casuse the blocking, but they are keep on coming.

Does anybody have any idea on what casuses this problem or a teporary solution for this? Please help.

Thx

you need to findout what these process is actually doing.. run

dbcc inputbuffer(185) to see the sql statement which these spid is executing.

madhu