The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information. ERROR MSG 8623
This Error was received by one of the developer on his Development Server. The first impression was, Developer has must have written a extremely complex Long queries to meet complex Business requirement.
But Actually he was just running a simple SELECT statement, like this
SELECT Column 1 , Column 2 ,Column 3 from Table 1 where Column 1 in (Select DISTINCT Column1 from Table 2 where COLUMN2 = @userVAR)
It’s hard to digest that this is a complex Query and we were able to reproduce this issue by running this again and again.
Just for information, The tables used in above illustration has huge data. Table 1 has approx. .9 million records and Table has approx .14 million records
My objective was to get away from this Message and User should able to get the results because I was running out of time.
SOLUTION / FIX :
I suggested developer to Join the tables instead of using the IN clause, which immediately FIXED this issue.
A day after, I was spending time to analyze why this SQL server treated that simple Query as extremely COMPLEX QUERY and retuned an Error Message. I started running this query on server by changing where clause condition and I was surprised with result.
I was able to get this error again when attempting to select records through a sub query query with a large number of entries in the “IN” clause. The moment I sub query started returning less records there was no Error/ Warning message.
Recommendation : If you have left out with no other option and have to use IN CLAUSE only with large table then
- Simplify your query
- may divide query into multiple sub query (DIVIDE and RULE)
- modify the Query plan bu supplying HINTS.
LESSON LEARNED :
In case you are using a sub query to extract the records from other table. THE IN clause should have less than 10,000 records (< 10,000), if its returning more than that you might get I was getting error “ran out of internal resources and could not produce a query plan“. Later on I found the same thing in Microsoft Documentation about this IN clause limitation.
This once again confirmed that Joining table is better option that using a IN CLAUSE.