[unixODBC-support] Re: Chinese encoding issue

Martin J. Evans martin.evans at easysoft.com
Fri Feb 29 20:45:58 GMT 2008


Yu-Hui Jin wrote:
> Martin,
>
> I finally made it work.  Just as you said, the issue is at the FreeTDS 
> side. It turned out i have to set the
> client charset = GB2312
> in the freetds.conf file for each server section.  Thanks for the help.
>
>
>
Glad you worked it out. I guess this should be a freetds faq.

Martin
> regards,
>
> -Hui
>
>
>
>
> On Wed, Feb 27, 2008 at 3:19 PM, Yu-Hui Jin <yuhui.jin at gmail.com 
> <mailto:yuhui.jin at gmail.com>> wrote:
>
>     Thanks, Martin. 
>
>     Yes, I found the iusql utility. The interesting thing is when I
>     ran the SQL statement from isql, it returns the rows only that the
>     Chinese are garbled.  When I ran the same statement in iusql, 
>     there's no data returned. The column headers are returned
>     correctly, but no data (it says SQLRowCount returns -1).  In fact,
>     for any SQL queries I issued under iusql,  there's always empty
>     results.   I don't know whether this is also caused by my not
>     setting up iconv.
>
>     The display of my console should be set up correctly to display
>     GB2312, since I have no problem seeing the Chinese characters
>     displayed by the tsql command line tool (from FreeTDS). 
>
>     Now regarding installing iconv,  does that mean I should follow
>     the steps:
>
>     1. download iconv from the web, compile and install the lib
>
>     My ruby seems to have the "iconv" library by default when it's
>     installed, but I don't know how ruby finds it. Is there a way to
>     find out whether and where I have iconv library installed on my host?
>
>
>     2. recompile unixODBC with some configs set to find the iconv.  I
>     see there are three related configs:
>
>       --enable-iconv          build with iconv support default=yes
>       --with-iconv-char-enc=enc   Encoding to use as ASCII
>     default=auto-search
>       --with-iconv-ucode-enc=enc  Encoding to use as UNICODE
>     default=auto-search
>
>     Shall I run ./configure with two of them:   --enable-iconv 
>     --with-iconv-ucode-enc=enc  ?
>
>
>     Sorry for so many email.  Many thanks for your prompt reply.
>
>
>     regards,
>
>     Yuhui
>
>
>
>     On Wed, Feb 27, 2008 at 2:53 PM, Martin J. Evans
>     <martin.evans at easysoft.com <mailto:martin.evans at easysoft.com>> wrote:
>
>         Yu-Hui Jin wrote:
>         > Hi, Martin,
>         >
>         > I searched on the web and it seems the utility you are
>         talking about
>         > is "osql", a unicode compliant tool that has a similar
>         interface as
>         > isql.  However, it is a Window 32 command line tool but the
>         OS I am
>         > working on is  RedHat Enterprise Linux 3.
>         If you build a recent(ish) unixODBC from source you will get two
>         binaries: isql and uisql or iusql (I can't remember off hand).
>         Just do a
>         which isql to find isql then do ls <dir_where_isql_found>*sql* and
>         you'll find it.
>         >
>         > But then what I really need is not to run isql or osql. I
>         ran them in
>         > hope of finding out whether unixODBC returns me the correct
>         encoding
>         > for the Chinese data.  But as you pointed out, the isql
>         utility can't
>         > do it.  So let me go straight to what I have tested in my
>         scripts.
>         >
>         > I set up the unixODBC and FreeTDS, etc. according to the
>         online guide:
>         >
>         http://wiki.rubyonrails.org/rails/pages/HowtoConnectToMicrosoftSQLServerFromRailsOnLinux
>         >
>         > I then tried two ways to connect to the SQL Server 2000 in
>         my scripts
>         > and both succeeded connecting and retrieving data back yet
>          failed to
>         > read the Chinese data in the correct encoding.  The query
>         returned
>         > correctly the numbers and English data, but the Chinese data
>         encoded
>         > in GB2312 were all garbled.  The two ways i tried are:
>         >
>         > 1. in a ruby script, i have the driver set up like:
>         >
>         > #!/usr/local/bin/ruby
>         > require 'dbi'
>         >
>         > DBI.connect('DBI:ODBC:testdb', 'username', 'password') do |
>         dbh |
>         >   dbh.select_all('select TOP 10 * from ATABLE with(nolock)')
>         do | row |
>         >    row.each_with_name do |val, name|
>         >      puts "#{name}, #{val.to_s}"
>         >    end
>         >    print "\n"
>         >   end
>         > end
>         >
>         > Some "name" data are in Chinese, but all of them are garbled.
>         >
>         >
>         > 2. in my Rails app, I have set up sql server in the
>         database.yml as
>         > follows :
>         >
>         > # Linux
>         > development:
>         >   adapter: sqlserver
>         >   mode: odbc
>         >   dsn: testcoredb
>         >   username: sa
>         >   password: joyo
>         > ....
>         > ....
>         >
>         > All Chinese data retrieved are garbled.
>         >
>         > Since runing tsql returns correct Chinese data, I'm suspecting
>         > somewhere unixODBC messed up with the Chinese data.  But it
>         could also
>         > be I'm not installing unixODBC with the right configuration.
>         >
>         > Any help or suggestions are much appreciated. Let me know if
>         you need
>         > me to provide more info.
>         >
>         >
>         > Thanks,
>         >
>         > Yuhui
>         >
>         >
>
>         unixODBC uses iconv so you need to set up iconv and your
>         environment
>         variables correctly before you'll get you data back properly.
>         I don't
>         use freeTDS. Someone on the freeTDS mailing list must have
>         done this
>         before. Also, bare in mind that printing from ruby or anything
>         else is
>         subject to display in your console/terminal and that needs to
>         understand
>         the unicode characters too.
>
>         Martin
>         _______________________________________________
>         unixODBC-support mailing list
>         unixODBC-support at easysoft.com
>         <mailto:unixODBC-support at easysoft.com>
>         http://mail.easysoft.com/mailman/listinfo/unixodbc-support
>
>
>
>
>     -- 
>     Regards,
>
>     -Hui 
>
>
>
>
> -- 
> Regards,
>
> -Hui
> ------------------------------------------------------------------------
>
>   




More information about the unixODBC-support mailing list