[unixODBC-support] Re: Chinese encoding issue

Yu-Hui Jin yuhui.jin at gmail.com
Wed Feb 27 23:19:31 GMT 2008


Thanks, Martin.

Yes, I found the iusql utility. The interesting thing is when I ran the SQL
statement from isql, it returns the rows only that the Chinese are garbled.
When I ran the same statement in iusql,  there's no data returned. The
column headers are returned correctly, but no data (it says SQLRowCount
returns -1).  In fact, for any SQL queries I issued under iusql,  there's
always empty results.   I don't know whether this is also caused by my not
setting up iconv.

The display of my console should be set up correctly to display GB2312,
since I have no problem seeing the Chinese characters displayed by the tsql
command line tool (from FreeTDS).

Now regarding installing iconv,  does that mean I should follow the steps:

1. download iconv from the web, compile and install the lib

My ruby seems to have the "iconv" library by default when it's installed,
but I don't know how ruby finds it. Is there a way to find out whether and
where I have iconv library installed on my host?


2. recompile unixODBC with some configs set to find the iconv.  I see there
are three related configs:

  --enable-iconv          build with iconv support default=yes
  --with-iconv-char-enc=enc   Encoding to use as ASCII default=auto-search
  --with-iconv-ucode-enc=enc  Encoding to use as UNICODE default=auto-search

Shall I run ./configure with two of them:   --enable-iconv
--with-iconv-ucode-enc=enc  ?


Sorry for so many email.  Many thanks for your prompt reply.


regards,

Yuhui


On Wed, Feb 27, 2008 at 2:53 PM, Martin J. Evans <martin.evans at easysoft.com>
wrote:

> Yu-Hui Jin wrote:
> > Hi, Martin,
> >
> > I searched on the web and it seems the utility you are talking about
> > is "osql", a unicode compliant tool that has a similar interface as
> > isql.  However, it is a Window 32 command line tool but the OS I am
> > working on is  RedHat Enterprise Linux 3.
> If you build a recent(ish) unixODBC from source you will get two
> binaries: isql and uisql or iusql (I can't remember off hand). Just do a
> which isql to find isql then do ls <dir_where_isql_found>*sql* and
> you'll find it.
> >
> > But then what I really need is not to run isql or osql. I ran them in
> > hope of finding out whether unixODBC returns me the correct encoding
> > for the Chinese data.  But as you pointed out, the isql utility can't
> > do it.  So let me go straight to what I have tested in my scripts.
> >
> > I set up the unixODBC and FreeTDS, etc. according to the online guide:
> >
> http://wiki.rubyonrails.org/rails/pages/HowtoConnectToMicrosoftSQLServerFromRailsOnLinux
> >
> > I then tried two ways to connect to the SQL Server 2000 in my scripts
> > and both succeeded connecting and retrieving data back yet  failed to
> > read the Chinese data in the correct encoding.  The query returned
> > correctly the numbers and English data, but the Chinese data encoded
> > in GB2312 were all garbled.  The two ways i tried are:
> >
> > 1. in a ruby script, i have the driver set up like:
> >
> > #!/usr/local/bin/ruby
> > require 'dbi'
> >
> > DBI.connect('DBI:ODBC:testdb', 'username', 'password') do | dbh |
> >   dbh.select_all('select TOP 10 * from ATABLE with(nolock)') do | row |
> >    row.each_with_name do |val, name|
> >      puts "#{name}, #{val.to_s}"
> >    end
> >    print "\n"
> >   end
> > end
> >
> > Some "name" data are in Chinese, but all of them are garbled.
> >
> >
> > 2. in my Rails app, I have set up sql server in the database.yml as
> > follows :
> >
> > # Linux
> > development:
> >   adapter: sqlserver
> >   mode: odbc
> >   dsn: testcoredb
> >   username: sa
> >   password: joyo
> > ....
> > ....
> >
> > All Chinese data retrieved are garbled.
> >
> > Since runing tsql returns correct Chinese data, I'm suspecting
> > somewhere unixODBC messed up with the Chinese data.  But it could also
> > be I'm not installing unixODBC with the right configuration.
> >
> > Any help or suggestions are much appreciated. Let me know if you need
> > me to provide more info.
> >
> >
> > Thanks,
> >
> > Yuhui
> >
> >
>
> unixODBC uses iconv so you need to set up iconv and your environment
> variables correctly before you'll get you data back properly. I don't
> use freeTDS. Someone on the freeTDS mailing list must have done this
> before. Also, bare in mind that printing from ruby or anything else is
> subject to display in your console/terminal and that needs to understand
> the unicode characters too.
>
> Martin
> _______________________________________________
> unixODBC-support mailing list
> unixODBC-support at easysoft.com
> http://mail.easysoft.com/mailman/listinfo/unixodbc-support
>



-- 
Regards,

-Hui
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.unixodbc.org/pipermail/unixodbc-support/attachments/20080227/45445fe4/attachment.html>


More information about the unixODBC-support mailing list