[unixODBC-support] Re: Chinese encoding issue

Martin J. Evans martin.evans at easysoft.com
Wed Feb 27 22:53:17 GMT 2008


Yu-Hui Jin wrote:
> Hi, Martin,
>
> I searched on the web and it seems the utility you are talking about 
> is "osql", a unicode compliant tool that has a similar interface as 
> isql.  However, it is a Window 32 command line tool but the OS I am 
> working on is  RedHat Enterprise Linux 3.
If you build a recent(ish) unixODBC from source you will get two 
binaries: isql and uisql or iusql (I can't remember off hand). Just do a 
which isql to find isql then do ls <dir_where_isql_found>*sql* and 
you'll find it.
>
> But then what I really need is not to run isql or osql. I ran them in 
> hope of finding out whether unixODBC returns me the correct encoding 
> for the Chinese data.  But as you pointed out, the isql utility can't 
> do it.  So let me go straight to what I have tested in my scripts.
>
> I set up the unixODBC and FreeTDS, etc. according to the online guide:
> http://wiki.rubyonrails.org/rails/pages/HowtoConnectToMicrosoftSQLServerFromRailsOnLinux
>
> I then tried two ways to connect to the SQL Server 2000 in my scripts 
> and both succeeded connecting and retrieving data back yet  failed to 
> read the Chinese data in the correct encoding.  The query returned 
> correctly the numbers and English data, but the Chinese data encoded 
> in GB2312 were all garbled.  The two ways i tried are:
>
> 1. in a ruby script, i have the driver set up like:
>
> #!/usr/local/bin/ruby
> require 'dbi'
>  
> DBI.connect('DBI:ODBC:testdb', 'username', 'password') do | dbh |
>   dbh.select_all('select TOP 10 * from ATABLE with(nolock)') do | row |
>    row.each_with_name do |val, name|
>      puts "#{name}, #{val.to_s}"
>    end
>    print "\n" 
>   end
> end
>
> Some "name" data are in Chinese, but all of them are garbled.
>
>
> 2. in my Rails app, I have set up sql server in the database.yml as 
> follows :
>
> # Linux
> development:
>   adapter: sqlserver
>   mode: odbc
>   dsn: testcoredb
>   username: sa
>   password: joyo
> ....
> ....
>
> All Chinese data retrieved are garbled.
>
> Since runing tsql returns correct Chinese data, I'm suspecting 
> somewhere unixODBC messed up with the Chinese data.  But it could also 
> be I'm not installing unixODBC with the right configuration.  
>
> Any help or suggestions are much appreciated. Let me know if you need 
> me to provide more info.
>
>
> Thanks,
>
> Yuhui
>
>

unixODBC uses iconv so you need to set up iconv and your environment 
variables correctly before you'll get you data back properly. I don't 
use freeTDS. Someone on the freeTDS mailing list must have done this 
before. Also, bare in mind that printing from ruby or anything else is 
subject to display in your console/terminal and that needs to understand 
the unicode characters too.

Martin



More information about the unixODBC-support mailing list