Avatar of thomasbau65
thomasbau65

asked on 

java sql resultSet max size

Hi all
I have following code:

 
public void getConstrains() throws SQLException
    {
        Statement st0 = conn.createStatement();
        ResultSet res0 = st0.executeQuery("SELECT CONSTRAINT_NAME "
                    + "FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS "
                    + "WHERE TABLE_NAME = '"+tName+"'");
        while(res0.next()){
            String cName = (String)res0.getObject(1);
            
            Statement st1 = conn.createStatement();
            st1.setMaxFieldSize(0x4000);
            ResultSet res1 = st1.executeQuery("SELECT CHECK_CLAUSE, CONSTRAINT_SCHEMA "
                    + "FROM INFORMATION_SCHEMA.CHECK_CONSTRAINTS "
                    + "WHERE CONSTRAINT_NAME='"+cName+"'");
            while(res1.next())
            {
                
//                java.io.InputStream ip = res1.getAsciiStream(1);
//                byte[] buff = new byte[1024];
//                String data = null;
//                for (;;) {
//                    try {
//                        int size = ip.read(buff);
//                        if (size == -1)
//                            break;
//                        data = new String(buff, 0, size);
//                    } catch (IOException ex) {
//                        Logger.getLogger(ConstraintsReader.class.getName()).log(Level.SEVERE, null, ex);
//                    }
//                }
//                 parse(data, (String)res1.getObject(2));
                parse((String)res1.getObject(1), (String)res1.getObject(2));
            }
        }
    }

Open in new window


When retrieving the constraints (res1)  I get something like: "([effctv_dir_angle]>=(0) AND [effctv_dir_angle]<=(359.9999))"  for some constraints the returned string is too long and gets cut to something like: "[effctv_dir_angle]>=(0) AND [effctv_dir_angle"

As you can see I tried to enlarge the ResultSet by setting "st1.setMaxFieldSize(0x4000)" and tried to have the response streamed without luck

Any body an idea about what I could do to resolve this problem

Thanks for helping out
th*
Java App ServersMicrosoft SQL ServerJava

Avatar of undefined
Last Comment
thomasbau65
Avatar of EugeneZ
EugeneZ
Flag of United States of America image

what driver are you using to access sql server?
what is your sql server version?
check thiese  articles :
http://msdn.microsoft.com/en-us/library/ms378813.aspx
http://msdn.microsoft.com/en-us/library/ms378599.aspx
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

>>
 for some constraints the returned string is too long and gets cut to something like: "[effctv_dir_angle]>=(0) AND [effctv_dir_angle"
>>

Why aren't you just doing the following?
String checkClause = res1.getString(1);

Open in new window

SOLUTION
Avatar of thomasbau65
thomasbau65

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

>>
@CEHJ
Should make no difference
>>

It will make all the difference ;)

a. you're not reading all the value
b. you're making reading the value unnecessarily complex
Avatar of thomasbau65
thomasbau65

ASKER

Well I'm not sure I understood all this correctly
I tried following thing:
 
Clob b = res1.getClob(1);
//b.getAsciiStream()getCharacterStream()
 //java.io.InputStream ip = res1.getAsciiStream(1);
Reader reader = b.getCharacterStream();
StringBuilder buffer = new StringBuilder();
char[] buff0 = new char[24576];
try {
    while(reader.read() > 0)
    {
        reader.read(buff0);
        buffer.append(new String(buff0, 0, buff0.length));
    }
} catch (IOException ex) {
    Logger.getLogger(ConstraintsReader.class.getName()).log(Level.SEVERE, null, ex);
}

Open in new window

And come to the conclusion that the driver (jTDs) must be the trouble maker, because the buffer is really large enough and when run in the debugger mode I can see that it has been instantiated in the defined size. Additionally the while statement iterates just once.
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

What type of column is the first one?
SOLUTION
Avatar of thomasbau65
thomasbau65

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Did you try the suggestion i already made?
Avatar of thomasbau65
thomasbau65

ASKER

Oooh yes
before I wrote this question :-)
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

And what was the result?
Avatar of thomasbau65
thomasbau65

ASKER

the same as with getObject
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

In fact - let's short circuit this somewhat: please post the code you tried, along with the result
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

>>the same as with getObject

You're not using getObject in your code posting - you're using getAsciiStream
Avatar of thomasbau65
thomasbau65

ASKER

Ok
 
Here ist the connection:
Connection conn = DriverManager.getConnection("jdbc:jtds:sqlserver://localhost/MYDB", "sa", "password");

Try with more properties:
Connection conn = DriverManager.getConnection("jdbc:jtds:sqlserver://localhost/MYDB; bufferMaxMemory=12288; lobBuffer=16384", "sa", "password");

And here the method reading the resultset
 
public void getConstrains() throws SQLException
    {
        Statement st0 = conn.createStatement();
        ResultSet res0 = st0.executeQuery("SELECT CONSTRAINT_NAME "
                    + "FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS "
                    + "WHERE TABLE_NAME = '"+tName+"'");
        while(res0.next()){
            String cName = (String)res0.getObject(1);
            
            Statement st1 = conn.createStatement();
            st1.setMaxFieldSize(0x4000);
            String sql = "SELECT CHECK_CLAUSE, CONSTRAINT_SCHEMA "
                    + "FROM INFORMATION_SCHEMA.CHECK_CONSTRAINTS "
                    + "WHERE CONSTRAINT_NAME='"+cName+"'";
            st1.setCursorName(sql);
            ResultSet res1 = st1.executeQuery(sql);
            
            while(res1.next())
            {
                /*
                 * Try Blob and Clob
                 */
                Clob b = res1.getClob(1);
                Reader reader = b.getCharacterStream();
                StringBuilder buffer = new StringBuilder();
                char[] buff0 = new char[24576];
                try {
                    while(reader.read() > 0)
                    {
                        reader.read(buff0);
                        buffer.append(new String(buff0, 0, buff0.length));
                    }
                } catch (IOException ex) {
                    Logger.getLogger(ConstraintsReader.class.getName()).log(Level.SEVERE, null, ex);
                } 
                parse(buffer.toString(), res1.getString(2));
                /*
                 * Try with ascii stream
                 */
                java.io.InputStream ip = res1.getAsciiStream(1);
                byte[] buff1 = new byte[12288];
                String data = null;
                for (;;) {
                    try {
                        int size = ip.read(buff1);
                        if (size == -1)
                            break;
                        data = new String(buff1, 0, size);
                    } catch (IOException ex) {
                        Logger.getLogger(ConstraintsReader.class.getName()).log(Level.SEVERE, null, ex);
                    }
                }
                parse(data, res1.getString(2));
                /*
                 * Try with object
                 */
                parse((String)res1.getObject(1), (String)res1.getObject(2));
                /*
                 * Try with String
                 */
                parse(res1.getString(1), res1.getString(2));
            }
        }
    }

Open in new window


Avatar of gordon_vt02
gordon_vt02

The code you posted with the CLOB will skip characters in the stream.  Every time your while conditional (reader.read() > 0) is checked, you lose one character.  That loop should be:

int readCount;
while((readCount = reader.read(buff0)) > 0)
    {
        buffer.append(new String(buff0, 0, readCount));
    }

Open in new window


Your append was also incorrect as it would always append the full character buffer to your output string regardless of how many characters you actually read in.  Not sure that will help with your original problem, but it will be a better test of the CLOB.
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

What's the error with that code? You probably need to post the parse method too
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Yes of course, gordon is right - which is what i meant earlier by the fact you're not reading the whole field (in two different way and two different places). Also

a. you should be using a join
b. you should be closing your db resources
c. you should be using getString (mentioned earlier) - no reason that i can see for getting clobs or streams
Avatar of thomasbau65
thomasbau65

ASKER

@gordon_vt02
I corrected the while statement
Still the returned string is incomplete

@CEHJ
The parse method is ok and working for all other CHECK_CLAUSE
Adding this would only add noise and will not contribute to a solution finding

Yes the JOIN would be a better way, still running the statement
"SELECT CHECK_CLAUSE, CONSTRAINT_SCHEMA "
                    + "FROM INFORMATION_SCHEMA.CHECK_CONSTRAINTS "
                    + "WHERE CONSTRAINT_NAME=constraintID'"
 alone (without the primary one) is not resolving the problem. I still receive only a small part of the 111k Bytes I should receive


Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Try the following, and if it's still incomplete, it can be tweaked:
public void getConstraints(String tableName) throws SQLException {
        PreparedStatement st = null;
        ResultSet res = null;
        String sql = "select" + "CC.CHECK_CLAUSE, CC.CONSTRAINT_SCHEMA" +
            "from INFORMATION_SCHEMA.TABLE_CONSTRAINTS C" + "inner join" +
            "INFORMATION_SCHEMA.CHECK_CONSTRAINTS CC" + "" +
            "on C.CONSTRAINT_NAME = CC.CONSTRAINT_NAME" +
            "where C.TABLE_NAME = ?";

        try {
            st = conn.prepareStatement(sql);
            st.setString(1, tableName);
            res = st.executeQuery();

            while (res.next()) {
                parse(res.getString(1), res.getString(2));
            }
        } finally {
            try {
                res.close();
            } catch (Exception e) { /* ignore */
            }

            try {
                st.close();
            } catch (Exception e) { /* ignore */
            }
        }
    }

Open in new window

Avatar of thomasbau65
thomasbau65

ASKER

I tried it but the sql statement

"select CC.CHECK_CLAUSE, CC.CONSTRAINT_SCHEMA " +
            "from INFORMATION_SCHEMA.TABLE_CONSTRAINTS C inner join " +
            "INFORMATION_SCHEMA.CHECK_CONSTRAINTS CC " +
            "on C.CONSTRAINT_NAME = CC.CONSTRAINT_NAME " +
            "where C.TABLE_NAME = ?";

dose not retrieve any values from my db
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

What value did you give for the parameter?
Avatar of thomasbau65
thomasbau65

ASKER

I tried both the table name as well as the constraints script-id
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Well i can't see anything wrong with the sql, but i can't test it as i don't have Windows. Maybe someone else can?
Avatar of thomasbau65
thomasbau65

ASKER

Lets just concentrate on the real problem, which is

running following sql-statement=

"SELECT CHECK_CLAUSE, CONSTRAINT_SCHEMA "
                    + "FROM INFORMATION_SCHEMA.CHECK_CONSTRAINTS "
                    + "WHERE CONSTRAINT_NAME=constraintID'"
 
I still receive only a small part of the 111k Bytes I should receive

I can also state that the parser code as well as the connection inclusive closing of resultset an statement is ok

running this cosde
 
public void getConstraints(String tableName) throws SQLException {
        tName = tableName;
        PreparedStatement st = null;
        ResultSet res = null;
        
        String sql = "SELECT CHECK_CLAUSE, CONSTRAINT_SCHEMA "
                        + "FROM INFORMATION_SCHEMA.CHECK_CONSTRAINTS "
                        + "WHERE CONSTRAINT_NAME=?";

        try {
            st = conn.prepareStatement(sql);
            st.setString(1, tableName);
            res = st.executeQuery();

            while (res.next()) {
                parse(res.getString(1), res.getString(2));
            }
        } finally {
            try {
                res.close();
            } catch (Exception e) { /* ignore */
            }

            try {
                st.close();
            } catch (Exception e) { /* ignore */
            }
        }
    }

Open in new window


will crash the parser because of splitting on "[" and "]" and since the string is cut of and some point the parse method fail
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

OK - good idea. So

>>will crash the parser

you mean it DOES crash the parser?

Can you tell me what this prints?
while (res.next()) {
                String s1 = res.getString(1);
                String s2 = res.getString(2);
                System.out.printf("s1 length = %d, s2 length = %d\n", s1.length(), s2.length());
                parse(s1, s2);
            }

Open in new window

Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Once it's been established definitely that even the simple code above fails to read the value(s) completely, you can try the following: (see http://technojeeves.com/joomla/index.php/free/51-copying-streams for IOUtils source)
java.io.StringWriter out = new java.io.StringWriter();
	net.proteanit.io.IOUtils.copyReader(res.getCharacterStream(1), out);
        String s1 = out.toString();

Open in new window

Avatar of thomasbau65
thomasbau65

ASKER

The error of the parse method has nothing to do with the problem as such.
The parse error comes from the fact that the data retrieving is not proper. (I'm expecting pair of brackets and at some point only get one)
I can see that when debugging
The problem lies on the driver side or beyond
I guess I will try the mssql driver instead of jtds  
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Could be, yes
Avatar of thomasbau65
thomasbau65

ASKER

:-(
It is not !!
nor is the IOUtils class  helping

may be I have to file a bug report to microsoft :-)

Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

So what happened when you ran http:#36917405 ?
ASKER CERTIFIED SOLUTION
Avatar of CEHJ
CEHJ
Flag of United Kingdom of Great Britain and Northern Ireland image

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
Avatar of thomasbau65
thomasbau65

ASKER

non of the expert could help
Microsoft SQL Server
Microsoft SQL Server

Microsoft SQL Server is a suite of relational database management system (RDBMS) products providing multi-user database access functionality.SQL Server is available in multiple versions, typically identified by release year, and versions are subdivided into editions to distinguish between product functionality. Component services include integration (SSIS), reporting (SSRS), analysis (SSAS), data quality, master data, T-SQL and performance tuning.

171K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
Ask the experts
Read over 600 more reviews

TRUSTED BY

IBM logoIntel logoMicrosoft logoUbisoft logoSAP logo
Qualcomm logoCitrix Systems logoWorkday logoErnst & Young logo
High performer badgeUsers love us badge
LinkedIn logoFacebook logoX logoInstagram logoTikTok logoYouTube logo