[unixODBC-support] Re: Chinese encoding issue

Yu-Hui Jin yuhui.jin at gmail.com
Fri Feb 29 20:10:23 GMT 2008


Martin,

I finally made it work.  Just as you said, the issue is at the FreeTDS side.
It turned out i have to set the

client charset = GB2312

in the freetds.conf file for each server section.  Thanks for the help.



regards,

-Hui




On Wed, Feb 27, 2008 at 3:19 PM, Yu-Hui Jin <yuhui.jin at gmail.com> wrote:

> Thanks, Martin.
>
> Yes, I found the iusql utility. The interesting thing is when I ran the
> SQL statement from isql, it returns the rows only that the Chinese are
> garbled.  When I ran the same statement in iusql,  there's no data returned.
> The column headers are returned correctly, but no data (it says SQLRowCount
> returns -1).  In fact, for any SQL queries I issued under iusql,  there's
> always empty results.   I don't know whether this is also caused by my not
> setting up iconv.
>
> The display of my console should be set up correctly to display GB2312,
> since I have no problem seeing the Chinese characters displayed by the tsql
> command line tool (from FreeTDS).
>
> Now regarding installing iconv,  does that mean I should follow the steps:
>
>
> 1. download iconv from the web, compile and install the lib
>
> My ruby seems to have the "iconv" library by default when it's installed,
> but I don't know how ruby finds it. Is there a way to find out whether and
> where I have iconv library installed on my host?
>
>
> 2. recompile unixODBC with some configs set to find the iconv.  I see
> there are three related configs:
>
>   --enable-iconv          build with iconv support default=yes
>   --with-iconv-char-enc=enc   Encoding to use as ASCII default=auto-search
>   --with-iconv-ucode-enc=enc  Encoding to use as UNICODE
> default=auto-search
>
> Shall I run ./configure with two of them:   --enable-iconv
> --with-iconv-ucode-enc=enc  ?
>
>
> Sorry for so many email.  Many thanks for your prompt reply.
>
>
> regards,
>
> Yuhui
>
>
>
> On Wed, Feb 27, 2008 at 2:53 PM, Martin J. Evans <
> martin.evans at easysoft.com> wrote:
>
> > Yu-Hui Jin wrote:
> > > Hi, Martin,
> > >
> > > I searched on the web and it seems the utility you are talking about
> > > is "osql", a unicode compliant tool that has a similar interface as
> > > isql.  However, it is a Window 32 command line tool but the OS I am
> > > working on is  RedHat Enterprise Linux 3.
> > If you build a recent(ish) unixODBC from source you will get two
> > binaries: isql and uisql or iusql (I can't remember off hand). Just do a
> > which isql to find isql then do ls <dir_where_isql_found>*sql* and
> > you'll find it.
> > >
> > > But then what I really need is not to run isql or osql. I ran them in
> > > hope of finding out whether unixODBC returns me the correct encoding
> > > for the Chinese data.  But as you pointed out, the isql utility can't
> > > do it.  So let me go straight to what I have tested in my scripts.
> > >
> > > I set up the unixODBC and FreeTDS, etc. according to the online guide:
> > >
> > http://wiki.rubyonrails.org/rails/pages/HowtoConnectToMicrosoftSQLServerFromRailsOnLinux
> > >
> > > I then tried two ways to connect to the SQL Server 2000 in my scripts
> > > and both succeeded connecting and retrieving data back yet  failed to
> > > read the Chinese data in the correct encoding.  The query returned
> > > correctly the numbers and English data, but the Chinese data encoded
> > > in GB2312 were all garbled.  The two ways i tried are:
> > >
> > > 1. in a ruby script, i have the driver set up like:
> > >
> > > #!/usr/local/bin/ruby
> > > require 'dbi'
> > >
> > > DBI.connect('DBI:ODBC:testdb', 'username', 'password') do | dbh |
> > >   dbh.select_all('select TOP 10 * from ATABLE with(nolock)') do | row
> > |
> > >    row.each_with_name do |val, name|
> > >      puts "#{name}, #{val.to_s}"
> > >    end
> > >    print "\n"
> > >   end
> > > end
> > >
> > > Some "name" data are in Chinese, but all of them are garbled.
> > >
> > >
> > > 2. in my Rails app, I have set up sql server in the database.yml as
> > > follows :
> > >
> > > # Linux
> > > development:
> > >   adapter: sqlserver
> > >   mode: odbc
> > >   dsn: testcoredb
> > >   username: sa
> > >   password: joyo
> > > ....
> > > ....
> > >
> > > All Chinese data retrieved are garbled.
> > >
> > > Since runing tsql returns correct Chinese data, I'm suspecting
> > > somewhere unixODBC messed up with the Chinese data.  But it could also
> > > be I'm not installing unixODBC with the right configuration.
> > >
> > > Any help or suggestions are much appreciated. Let me know if you need
> > > me to provide more info.
> > >
> > >
> > > Thanks,
> > >
> > > Yuhui
> > >
> > >
> >
> > unixODBC uses iconv so you need to set up iconv and your environment
> > variables correctly before you'll get you data back properly. I don't
> > use freeTDS. Someone on the freeTDS mailing list must have done this
> > before. Also, bare in mind that printing from ruby or anything else is
> > subject to display in your console/terminal and that needs to understand
> > the unicode characters too.
> >
> > Martin
> > _______________________________________________
> > unixODBC-support mailing list
> > unixODBC-support at easysoft.com
> > http://mail.easysoft.com/mailman/listinfo/unixodbc-support
> >
>
>
>
> --
> Regards,
>
> -Hui




-- 
Regards,

-Hui
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.unixodbc.org/pipermail/unixodbc-support/attachments/20080229/9e9cf218/attachment.html>


More information about the unixODBC-support mailing list