[unixODBC-support] Memory leak in SQLFetch?

Craddock, Richard C cmi5 at cdc.gov
Tue Dec 14 19:47:00 GMT 2004


Alright, I am sorry to waste everyone's time but I think that I have
found the problem.  There was no memory leak in the program that I
posted.  It is instead related to file logging.  I turned off file
logging and no longer have the problem with shrinking system memory.  

But why does writing to the log file affect the system memory?  I
suppose the file system caches the file in memory.  But why isn't this
memory flushed after the write?

Thanks for everyone's help.

-Cameron

-----Original Message-----
From: unixodbc-support-bounces at easysoft.com
[mailto:unixodbc-support-bounces at easysoft.com] On Behalf Of Craddock,
Richard C
Sent: Tuesday, December 14, 2004 2:12 PM
To: Support for the unixODBC project
Subject: RE: [unixODBC-support] Memory leak in SQLFetch?

Hmm...

I ran it for the full 20160 using the ODBC - ODBC bridge.  Here is why I
concluded that there is a memory leak:

1) during testing I noticed that my system memory continued to shrink
2) I noticed that the system memory shrinks by ~3 MB every time I run
the program
3) If I comment out the SQLFetch then the system memory does not leak.

But now I am confused.

-Cameron

-----Original Message-----
From: unixodbc-support-bounces at easysoft.com
[mailto:unixodbc-support-bounces at easysoft.com] On Behalf Of Saunders,
Steve
Sent: Tuesday, December 14, 2004 2:02 PM
To: 'Support for the unixODBC project'
Subject: RE: [unixODBC-support] Memorleak in SQLFetch?

Cameron, 

Doesn't look like it is growing any at all after it initializes.  The
RSS
and VSZ counts stay steady.  

Did you run it for 101 integrations or more?

Looking at the system memory usage can be very misleading when trying
find
memory leaks.  ps, top and gtop and similar tools that get down to the
process level are better.







> -----Original Message-----
> From: unixodbc-support-bounces at easysoft.com
> [mailto:unixodbc-support-bounces at easysoft.com]On Behalf Of Craddock,
> Richard C
> Sent: Tuesday, December 14, 2004 1:49 PM
> To: Support for the unixODBC project
> Subject: RE: [unixODBC-support] Memory leak in SQLFetch?
> 
> 
> Steve,
> 
> I performed the test that you asked for and these are the 
> results.  What
> do they mean?
> 
> -Cameron
> -------------------------
> 
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  16   0  4276 1640 -      S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  16   0  4276 1640 -      S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:00
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 667797 S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 -      S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 4     0  5423  3825  15   0  4276 1668 -      S    pts/2      0:01
> ./testfetch
> [root at localhost file_upload]# ps l 5423
> F   UID   PID  PPID PRI  NI   VSZ  RSS WCHAN  STAT TTY        TIME
> COMMAND
> 
> -----Original Message-----
> From: unixodbc-support-bounces at easysoft.com
> [mailto:unixodbc-support-bounces at easysoft.com] On Behalf Of Saunders,
> Steve
> Sent: Tuesday, December 14, 2004 12:23 PM
> To: 'Support for the unixODBC project'
> Subject: RE: [unixODBC-support] Memory leak in SQLFetch?
> 
> This display's memory for the system.  
> 
> Better data would come from the 'ps l pid' (where pid is the 
> process id
> of
> the program).  I would do it by getting the initial memory 
> usage of the
> program by having it pause for user input before it loops, then get a
> few
> different samples while it is running, and then have it pause for user
> input
> again before it exits and collect the memory usage of the 
> program before
> stopping it.  
> 
> This will give you the detail about how much memory the 
> specific program
> is
> using.  
> 
> This is what I watch for leaks with during load testing our 
> software on
> various Unix/Linux platforms.
> 
> 
> 
> 
> > -----Original Message-----
> > From: unixodbc-support-bounces at easysoft.com
> > [mailto:unixodbc-support-bounces at easysoft.com]On Behalf Of Craddock,
> > Richard C
> > Sent: Tuesday, December 14, 2004 12:15 PM
> > To: Support for the unixODBC project
> > Subject: RE: [unixODBC-support] Memory leak in SQLFetch?
> > 
> > 
> > Here is the latest information that I have discovered.
> > 
> > In order to detect memory leaks I run free -b before and 
> > after executing
> > my code.  I assume that the difference between the two was 
> > allocated but
> > not freed by the application.
> > 
> > If I execute the SQLfetch() loop 101 times I get the following data:
> > 
> > [root at localhost file_upload]# free -b
> >              total       used       free     shared    buffers
> > cached
> > Mem:    1326329856  604139520  722190336          0  281374720
> > 231182336
> > -/+ buffers/cache:   91582464 1234747392
> > Swap:   4104372224          0 4104372224
> > [root at localhost file_upload]# ./testfetch
> > [root at localhost file_upload]# free -b
> >              total       used       free     shared    buffers
> > cached
> > Mem:    1326329856  604270592  722059264          0  281391104
> > 231198720
> > -/+ buffers/cache:   91680768 1234649088
> > Swap:   4104372224          0 4104372224
> > 
> > This shows a reproducible difference of 131072 bytes between runs.
> > 
> > If I allow the SQLfetch() loop execute 11 times I get the following:
> > 
> > [root at localhost file_upload]# free -b
> >              total       used       free     shared    buffers
> > cached
> > Mem:    1326329856  604532736  721797120          0  281628672
> > 231239680
> > -/+ buffers/cache:   91664384 1234665472
> > Swap:   4104372224          0 4104372224
> > [root at localhost file_upload]# ./testfetch
> > [root at localhost file_upload]# free -b
> >              total       used       free     shared    buffers
> > cached
> > Mem:    1326329856  604532736  721797120          0  281636864
> > 231243776
> > -/+ buffers/cache:   91652096 1234677760
> > Swap:   4104372224          0 4104372224
> > 
> > There is no difference between the two.  Maybe there is a memory
> > corruption involved?  Attached is the sql.log file for the 
> > 100 run case.
> > 
> > Thanks,
> > Cameron
> > 
> > 
> _______________________________________________
> unixODBC-support mailing list
> unixODBC-support at easysoft.com
> http://mail.easysoft.com/mailman/listinfo/unixodbc-support
> 
> 
> 
> _______________________________________________
> unixODBC-support mailing list
> unixODBC-support at easysoft.com
> http://mail.easysoft.com/mailman/listinfo/unixodbc-support
> 
_______________________________________________
unixODBC-support mailing list
unixODBC-support at easysoft.com
http://mail.easysoft.com/mailman/listinfo/unixodbc-support



_______________________________________________
unixODBC-support mailing list
unixODBC-support at easysoft.com
http://mail.easysoft.com/mailman/listinfo/unixodbc-support






More information about the unixODBC-support mailing list