Authority data import fail (UTF-8)

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Authority data import fail (UTF-8)

Partha Mukhopadhyay

 

I was tarying last nigth importing authority data in koha (Koha 3.04.05.000 on Ubuntu LTS 10.04.02) through command line script. Let me report you the result-

 

1> Non-Unicode records......

---------------------------------

./bulkauthimport.pl -file /home/psm/Desktop/auth/auth_all_en.mrc
...
3 MARC record done in 0.0512549877166748 seconds

2> Unicode records......

---------------------------------

./bulkauthimport.pl -file /home/psm/Desktop/auth/auth_all_bn.mrc
utf8 "\xE0" does not map to Unicode at /usr/lib/perl/5.10/Encode.pm line 174, <GEN12> line 1.

What is happening?

----------------------------------------------------

Dr. Parthasarathi Mukhopadhyay

Lecturer (Sr. Scale)

Department of Library and Information Science

The University of Burdwan (WB)

------------------------------------------


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Authority data import fail (UTF-8)

MJ Ray-2
Partha Mukhopadhyay <[hidden email]>
> ./bulkauthimport.pl -file /home/psm/Desktop/auth/auth_all_bn.mrc  utf8 "\xE0" does not map to Unicode at /usr/lib/perl/5.10/Encode.pm line 174, &lt;GEN12&gt; line 1.
> What is happening?

Something is not working ;-)

Is it possible for you to reply to the list with the output of the
command "locale" and if that says UTF-8 a lot, can you prepare a
sample authority file containing one record that shows the problem and
put it somewhere for download, please?

Hope that helps,
--
MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op.
http://koha-community.org supporter, web and LMS developer, statistician.
In My Opinion Only: see http://mjr.towers.org.uk/email.html
Available for hire for Koha work http://www.software.coop/products/koha
_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/
Reply | Threaded
Open this post in threaded view
|

Re: Authority data import fail (UTF-8)

Partha Mukhopadhyay

Dear MJR

Ya I set locale as utf. See the result of locale command

 

psm@libra2:~$ locale
LANG=en_IN.utf8
LC_CTYPE="en_IN.utf8"
LC_NUMERIC="en_IN.utf8"
LC_TIME="en_IN.utf8"
LC_COLLATE="en_IN.utf8"
LC_MONETARY="en_IN.utf8"
LC_MESSAGES="en_IN.utf8"
LC_PAPER="en_IN.utf8"
LC_NAME="en_IN.utf8"
LC_ADDRESS="en_IN.utf8"
LC_TELEPHONE="en_IN.utf8"
LC_MEASUREMENT="en_IN.utf8"
LC_IDENTIFICATION="en_IN.utf8"
LC_ALL=
psm@libra2:~$

Here the command I used.....

psm@libra2:~$ export KOHA_CONF=/etc/koha/koha-conf.xml
psm@libra2:~$ export PERL5LIB=/usr/share/koha/lib
psm@libra2:~$ ./bulkauthimport.pl -file /home/psm/Desktop/auth/auth_all_bn.mrc
bash: ./bulkauthimport.pl: No such file or directory
psm@libra2:~$ cd /usr/share/koha/bin/migration_tools/
psm@libra2:/usr/share/koha/bin/migration_tools$ ./bulkauthimport.pl -file /home/psm/Desktop/auth/auth_all_bn.mrc
utf8 "\xE0" does not map to Unicode at /usr/lib/perl/5.10/Encode.pm line 174, <GEN12> line 1.
psm@libra2:/usr/share/koha/bin/migration_tools$

Here is the file content.....

 

01941cz  a2200361n  4500001000800000005001700008008004100025010001700066035002300083040005100106100002500157400003100182400002400213400003000237400002600267400002500293400004000318400004500358400002900403400002500432400004400457670004100501670005700542670012100599670009100720670011400811670008300925670010701008670014001115670029901255952001001554953001501564326270620090323092934.0800522n| acannaabn          |a aaa        an  50012860   a(OCoLC)oca00048347  aDLCbengcDLCdDLCdDLC-ONdNICdNNCdNcUdDLC1 সেন, অমর্ত্যd ১৯৩৩- 1 aSen, Amartya,d1933-1 aSen, Amartya Kumar,d1933-0 aAmartya Sen,d1933-0 aAmartya Kumar Sen,d1933-1 aSena, Amartya,d1933-0 aAmartya Sena,d1933-1 aSen, A. K.q(Amartya Kumar),d1933-1 aSen, Amartya K.q(Amartya Kumar),d1933-0 aʻAmāttaya Sen,d1933-1 aSen, Armatya,d1933-1 aSen, Amarty K.q(Amartya Kumar),d1933-  aAuthor's Choice of techniques, 1960.  aUtilitarianism and beyond, 1982:bt.p. (Amartya Sen)  aNobela jaẏī Amartya Sena, 1998:bt.p. (Amartya Sena) p. 170 (b. Nov. 3, 1933) p. 174 (recd. Nobel Prize in 1998)  aAuthor's Collective choice and social welfare, 1984, c1979:bCIP t.p. (Amartya K. Sen)  anuc88-94543: Als Keynes nu geleefd zou hebben, c1983b(hdg. on CU rept.: Sen, Amartya Kumar; usage: A.K. Sen)  aTalāt prachāthippatai læ kānphatthanā, 2000:bt.p. (ʻAmāttaya Sen)  aPerspectives on the economic and human development of India and China, 2006:bt.p. (Armatya [sic] Sen)  aCollective choice and social welfare, 1979:bt.p. (Amarty K. Sen) erratum (correct spelling of first name is Amartya instead of Amarty)  aBiog. resource center (Contemp. authors), Mar. 23, 2009b(Amartya Sen; also known as Amartya Kumar Sen, Amartya K. Sen, A.K. Sen; b. Nov. 3, 1933, Santiniketan, India; Trinity College, Cambridge, Ph. D., 1959; economist, educator, writer, and editor; Trinity College, Cambridge, master, 1998- )  aRETRO  axx00btd07
00420cz  a2200133n  4500001000800000005001700008008004100025010001700066035002100083040001400104100001700118670014200135953000900277249605119870716075403.8870526n| acannaab           |n aaa     b  an  86083777   a(DLC)n  86083777  aDNLMcDLC10aSen, Amar K.  aReceptors and ligands in neurological disorders, 1987:bCIP t.p. (Amar K. Sen, prof., Dept. of Pharm., Univ. of Toronto, Toronto, Canada)  abz57
00673cz  a2200157n  450000100070000000500170000700800410002401000170006503500210008204000180010310000290012140000200015067001580017067001720032895300150050014573420090303095926.0860925n| acannaab           |a aaa        an  85211318   a(DLC)n  85211318  aDLCcDLCdDLC1 aDas, A. K.q(Asok Kumar)1 aDas, Asok Kumar  aTikader, B. K. Glimpses of animal life of Andaman & Nicobar Islands, 1985:bt.p. (A.K. Das; Zoological Survey of India) jkt. (Dr. Asok Kumar Das; Ph. D.)  aA monograph of the flagellate symbiotes of Indian termites with special reference to their host correlation and coevolution, 2007:bt.p. (A.K. Das, emeritus scientist)  awd03bcg09

 

I'm attaching this file to you mail ID for further investigation.

 

Thanks and regards

Partha

----------------------------------------------------

Dr. Parthasarathi Mukhopadhyay

Lecturer (Sr. Scale)

Department of Library and Information Science

The University of Burdwan (WB)

------------------------------------------


At 22 Oct 2011 19:01:55 +0200 (CEST) from MJ Ray <[hidden email]>:

Partha Mukhopadhyay <[hidden email]>
> ./bulkauthimport.pl -file /home/psm/Desktop/auth/auth_all_bn.mrc utf8 "\xE0" does not map to Unicode at /usr/lib/perl/5.10/Encode.pm line 174, &lt;GEN12&gt; line 1.
> What is happening?

Something is not working ;-)

Is it possible for you to reply to the list with the output of the
command "locale" and if that says UTF-8 a lot, can you prepare a
sample authority file containing one record that shows the problem and
put it somewhere for download, please?

Hope that helps,
--
MJ Ray (slef), member of www.software.coop, a for-more-than-profit co-op.
http://koha-community.org supporter, web and LMS developer, statistician.
In My Opinion Only: see http://mjr.towers.org.uk/email.html
Available for hire for Koha work http://www.software.coop/products/koha
_______________________________________________
Koha-devel mailing list
Koha-devel@lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/


_______________________________________________
Koha-devel mailing list
[hidden email]
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/