[unixODBC-support] odbc performance problem (was ./configure is taking forever)

Reza Taheri rtaheri at vmware.com
Mon Feb 6 05:29:45 GMT 2012

Hi Nick,
Thanks a lot for all your patience and guidance. I got much better behavior by switching to unixODBC version 2.3.1 and psqlodbc version 09.01.0100. FWIW, my earlier reluctance to update to the latest version was due to the mistaken assumption that I'd have better luck with  version 2.2.14 of unixODBC and version 8.04 of psqlodbc because those are the versions that get shipped with RHEL 6.1.

I did run into one issue: I didn't get the *S.so libraries. After some digging, I realized that the SUBDIR definition in DRVConfig/Makefile was commented out. Uncommenting it fixed the problem. Just curious why that was so.

Now that I am able to build the product, I can get going on my real task!! I  am a member of the TPC (are you familiar with the TPC benchmarks?), and we are embarking on a major effort to use PGSQL to develop a reference kit for the TPC-E and TPC-V benchmarks. The benchmarking kit will use ODBC to make it easy for someone to take it and replace PGSQL with the database of their choice. We have run into a performance problem in the odbc library, which sent me down this path of getting the source and building the libraries from scratch. After changing the Makefiles to compile with -pg, I can see that the bottleneck is inside the functions __check_stmt_from_dbc and __set_stmt_from_dbc. This is what perf report -g gives me:

# Events: 18K cycles
# Overhead  Command        Shared Object                                                                                                       Symbol
# ........  .......  ...................  ...........................................................................................................
    74.72%     java  libodbc.so.2.0.0     [.] __check_stmt_from_dbc
               --- __check_stmt_from_dbc
                   CBrokerVolumeDB::DoBrokerVolumeFrame1(TPCE::TBrokerVolumeTxnInput const*, TPCE::TBrokerVolumeFrame1Output*)

    13.36%     java  libodbc.so.2.0.0     [.] __set_stmt_state
               --- __set_stmt_state
                   CBrokerVolumeDB::DoBrokerVolumeFrame1(TPCE::TBrokerVolumeTxnInput const*, TPCE::TBrokerVolumeFrame1Output*)

     1.66%     java  [kernel.kallsyms]    [k] default_send_IPI_mask_sequence_phys

Not only are we spending a lot of time in these functions, they appear to be serialized. So once throughput gets high enough to saturate one core with check_stmt_from_dbc/ set_stmt_from_dbc processing, we hit a wall. I guess I am going to be spending a lot more time getting familiar with the code!!


From: Nick Gorham [mailto:nick at lurcher.org]
Sent: Sunday, February 05, 2012 1:42 AM
To: Reza Taheri; Support for the unixODBC project
Subject: Re: [unixODBC-support] ./configure is taking forever

On 05/02/2012 03:11, Reza Taheri wrote:
Strace was a good idea. This is what I saw:

-          Using strace on odbcinst -q, saw that we were missing the dbcinst.ini file. How is that possible? Well, we are picking up the odbcinst commands from /usr/local/bin instead of /usr/bin. The new command looks for odbcinst.ini in /usr/local/etc, and that file is empty.

o   Copy /etc/odbcinst.ini /usr/local/etc

o   Make these changes:
Driver64      = /usr/local/lib/psqlodbcw.so
Setup64               = /usr/local/lib/libodbcpsqlS.so

o   odbcinst now runs OK. But we still get error messages from the SUT process:
DBConnector: Failed to connect

The driver reported the following diagnostics whilst running SQLDriverConnect

01000:1:0:[unixODBC][Driver Manager]Can't open lib '/usr/local/lib/psqlodbcw.so' : libodbcinst.so.2: cannot open shared object file: No such  file or directory

The problem is that psqlodbcw.so looks for libodbcinst.so.2, and that file is missing. When I make the unix odbc libraries, I get libodbcinst.so.1

I can fake this out, but I first tried to remake psqlodbc with LDFLAGS set to include /usr/local/lib, but now am running into compilations problems there :(.  Will continue tomorrow.

Thanks for listening!
Building current 2.3.1 or 2.3.2pre from the ftp site will give you the so.2 libs.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.unixodbc.org/pipermail/unixodbc-support/attachments/20120205/2b407d2a/attachment.html>

More information about the unixODBC-support mailing list