මේ රාජ්‍ය ආයතන මාර්ගයෙන් “ඊ-නිර්මාණකරුවන්ට” ප්‍රදානය කරනු ලබන්නේය කියන “සම්මාන” ගැන ලියැවෙන තෙවන සටහනය. මේ සටහන ලිවීමට ආසන්න හේතුව සිංහල බ්ලොග්කරුවන්ගේ සංසදයේ මිතුරන් එකිනෙකා විසින් තම තමන්ගේ බ්ලොග්වල පුන පුනා වහෙන් ඔරෝ බාසාවෙන් පළ කරනු ලබන නිවේදනයකි. මේ වන විට දස වතාවකට වඩා පුනරුච්චාරණය කොට ඇති මේ නිවේදනය උගුරට හොරා බෙහෙත් කෑමකි. ඒ ගැන ලේඛකයා පුදුම නොවේ. තමන් කොයිතරම් නිග්‍රහයට පත් කළද, තොරතුරු තාක්ෂණ නියෝජිතායතනය (Information and Communication Technology Agency හෙවත් ICTA) හා මේ සිද්ධියට ඍජුව වගකිවයුතු .lk වසම් අධිකාරිය බාර ආචාර්ය ගිහාන් ඩයස් තරහ නොකැරගෙන සිටීම යෙහෙකිය ඔවුන් කල්පනා කරනවා වන්නට පිළිවන. එවැනි බැඳීම් නැති හෙයින් කිව යුත්ත කෙළින්ම කීමටය මේ කෙටි සටහන.

රාජ්‍ය ආයතන මාර්ගයෙන් “ඊ-නිර්මාණකරුවන්ට” හෝ බ්ලොග්කරුවන්ට දෙනු ලබන්නේය කියන සම්මාන තනිකරම මගෝඩි ක්‍රියාවලියකි. මෙවන් “සම්මාන” දීම සඳහා මේ කිසිම ආයතනයකට සදාචාරාත්මක අයිතියක් නැත. අද සිංහල භාෂාවෙන් නිර්මාණය වෙමින් පවතින කිසිම ඩිජිටල් අන්තර්ගතයක් මොවුන්ගේ මැදිහත්වීමකින්, උත්තේජනයකින් ඇති වූවක් නොවේ.

Kottu.org හා “සිංහල බ්ලොග් කියවනය” වැනි සින්ඩිකේටරයන් බිහිව ඇත්තේ මේ කිසිම ආයතනයක සම්මාදමක් නැතිව තනිකරම පුද්ගලික උවමනාව මත තනි පුද්ගලයන්ගේ වියදමිනි. මලින්ත සමරකෝන්, බුද්ධික ජයසිංහ, කාලිංග ඇතුලත්මුදලි, සුපුන් සුදාරක ඇතුළු පිරිසක් සිංහල බ්ලොග් කියවනය නිර්මාණයෙහිද, ඉන්ඩි සමරජීව Kottu.org නිර්මාණය කිරීමෙහිද ප්‍රධානත්වයක් ගෙන කටයුතු කළයි අසා ඇත්තෙමි. (මේ නම් සඳහන් කෙරෙන්නේ ඔවුන් කළ සද්කාර්යය අගය කළයුතු නිසාය. මේ කිසිවකු මම නමින් මිස නොදනිමි. වරදක් ඇතොත් නිවැරදි කෙරෙත්වා!) සම්මාන දෙනවා නම් දිය යුත්තේ ඔවුන්ටය. මේ කර ඇති කාර්යයම ඊට ප්‍රමාණවත්ය. ඊට තව තරග අවශ්‍ය නොවේ.

විදේශාධාර මිලියන ගණනින් නාස්තිකරමින් තොරතුරු තාක්ෂණ නියෝජිතායතනය (ICTA) විසින් කරන ලද එකම කටයුත්ත යුනිකෝඩ් සම්මතය හඳුන්නා දීමක් පමණක් විය හැකිය. එහෙත් යුනිකෝඩයට බොහෝ ඉහත සිංහල අන්තර්ගතයන් (තිබස් බඳු වෙනත් ෆොන්ට භාවිතාවෙන්) බිහිවූ අතර කන්දක් විලි ලා වැදූ මී පැටවකු බඳු, තවමත් යංසයන් රේපයන් එල්ලා හැලෙන යුනිකෝඩය නැතිවද අද සිදුවෙමින් පවතින පරිවර්තනය වීමට ඉඩ තිබුණේය. (උදාහරණයක් හැටියට ඉතාමත් ඉහළ පාඨක ප්‍රතිචාරයක් ලබන ලංකාදීප, ලක්බිම හා ලංකා ඊ-නිව්ස් මංගත පුවත්පත් තවමත් යුනිකෝඩය භාවිතා නොකරයි.)

තොරතුරු තාක්ෂණ නියෝජිතායතනය හා .lk වසම් අධිකාරිය “ඊ-නිර්මාණකරුවන්ට”, බ්ලොග්කරුවන්ට “සම්මාන” දීමට මැදිහත් වන්නේ එකම හේතුවක් නිසාය. ඒ දැනට වර්ධනය වෙමින් පවතින සිංහල ඩිජිටල් අන්තර්ගතයන් පිළබඳ ගෞරවයෙන් කොටසක් තමන් ඩැහැ ගැනීම සඳහාය. මේ නිර්ලජ්ජිත ක්‍රියාවලියකි. තමන්ගේ කිසිම සම්මාදමකින් තොරව අනුන් කළ දෙයක ගෞරවය ගැනීමට කැස කවන්නෝ අපේ පිළිකුලට මිස ගෞරවයට පාත්‍ර වෙත්ද?

උන්ගේ ආච්චිලාගේ සම්මාන උන්ටම තියාගන්නට කියමු. අප මෙතෙක් කල් ඩිජිටල් අන්තර්ගතයන් නිර්මාණය කළේ උන්ගේ සම්මාන වෙනුවෙන්ද? නැත. අපි අපේ ගමන යමු. ඊට කාගෙන් හෝ කිහිලිකරු අවශ්‍ය නොවේ.

The answer is not so obvious as it was before.

In the time immemorial strings were ASCII strings, and since then we have UpperCase function. The function is  just replaces the lowercase ‘a’..’z’ characters by their uppercase ‘A’..’Z’ versions. Though nowadays the UpperCase is a Unicode function it still ‘uppercases’ only ASCII characters. That is good but that is not all uppercase functionality we need.

The early Delphi versions introduced ANSI strings. The uppercase conversion on ANSI strings includes local characters besides ASCII ‘a’..’z’ and consequntly is locale-dependent. The only locale supported by pre-unicode Delphi versions is system locale. The ANSIUpperCase function was introduced to uppercase ANSI strings using system locale. That is not so good as it may be, since sometimes we need a locale different from system locale, but at least is clear and unambiguous.

Now in Delphi2009 we can explicitly define the ANSI locale in Ansistring type, and that is great. The ANSI uppercase conversion is locale-sensitive, and now we can perform ANSI string uppercase conversions on any locale by just calling AnsiUpperCase function, right? The answer is – NO.

First, in Delphi 2009 the AnsiUpperCase is unicode function. I understand the [compatibility] reasons why the unicode function has Ansi prefix, but I prefer it would never happen.

Second, the real ANSI version of AnsiUpperCase function defined in Ansistrings unit is a wrapper for WinAPI CharUpperBuffA function, and still uses the current system locale and ignores the locale associated with the string type.

As a result it is better to avoid using AnsiUpperCase at all. The AnsiUpperCase function is just a backward compatibility issue.

Nowadays the defaul string type is Unicode string. The unicode string can include characters of all different locales at once, so the unicode strings are again locale-indendent as ASCII strings were, right? Well, it is almost right. But to be absolutely correct, the answer is still NO.

The only problem I know that makes Unicode uppercase conversion locale-dependent is the case of dotless i (ı, $0131) and dotted I (İ, $0130). In most languages the upper of i ($69) is I ($49), but in turkish locale i ($69) maps to İ ($0130). Similarly in turkish the lower of I ($49) is ı ($0131).

Now let us go back to Delphi. The new string format containes no locale information for unicode strings. We have 2 functions to make the unicode uppercase conversion – the already mentioned SysUtils.AnsiUpperCase introduced for backward compatibility and new ToUpper function defined in Character unit. The SysUtils.AnsiUpperCase is a wrapper for WinAPI CharUpperBuffW function and ignores locale-specific issues. The CharUpperBuffW works much better than its ANSI analog CharUpperBuffA, but still can’t help with “turkish case”. The Character.ToUpper function is a wrapper for LCMapString function and is locale-dependent, but the locale parameter is set to system locale. So if you need “turkish uppercase” on the system with different locale (or want to make the result of uppercase independent from system locale) you must write your own LCMapString wrapper.

BTW: the early Delphi 2009 releases containes side-effect bug in ToUpper and ToLower implementations. The bug was fixed in Update 3 (build 12.0.3420.21218).

I felt like getting my hands dirty in the unicode swamp, and actually writing myself a proper UTF-8 decoder.

I am putting this code public to argue a couple of points on good coding practice.

But as a backstory, i recommend the following about couroutines and why they are one of the dirtiest hacks that exists to date in ANSI C.

The actual code is small, and consistent. The ‘nasty’ goto statement for handling decoding errors I believe is a nice touch which in this case only contributes to clarity and consistency.

What do you think?

Khmer Unicode for Apple Mac computer

Nokorsoft will release Khmer Unicode MacOSX Lite V2.0. Khmer Unicode Mac OSX Lite enables Mac OS X users to read and write Khmer Unicode with most software on Mac OS machine.

What are in the package?

- In version 2.o, we removed the old Khmer Unicode font and replaced it by a Khmer Unicode font created  by Mr. Danh Hong, I do have his authorization to release this package.

- NiDA Khmer Keyboard Layout Version 1.0 & Version 2.0.

- Documents on how to use Khmer Unicode MacOSX can be found in Applications folder.

To get better Khmer Unicode fonts that will work with Microsoft Word and Adobe products, please contact Nokorsoft or Mr. Danh Hong via email: dnhhng@gmail.com

- Download Link:

Khmer Unicode MacOSX Lite Version 2.o will be available in near future.

Password to unlock zip file is: nokorsoft.com

- File Size: 889 KB

Per i non informatici: Unicode è un sistema di codifica che assegna un numero univoco ad ogni carattere usato per la scrittura di testi, in maniera indipendente dalla lingua, dalla piattaforma informatica e dal programma utilizzati (via Wikipedia).  In questo articolo potrete scoprire le meraviglie che contiene come ad esempio il carattere 0E5B (Khomut in Thai)

Un carattere elfico

Sin da quando lessi “Il signore degli anelli” ho la fissa della scrittura con le tengwar, cioè i caratteri elfici descritti in una delle appendici di quel romanzo. Esistono da anni vari strumenti (per vari sistemi operativi) per raggiungere questo scopo, ma tutti usano particolari font che mappano le lettere dell’alfabeto latino con le suddette tengwar. Invece il mio obiettivo era utilizzare i caratteri destinati a questo uso definiti nelle tabelle di Unicode, ammesso che ci fossero.

Perché? I motivi sono presto detti: le soluzioni esistenti presuppongono che chi scrive il documento elfico abbia installato almeno un font adatto (e che faccia lo stesso chi riceve il documento); inoltre la mappatura di cui sopra non è standard; e poi la codifica caratteri universale, dopo che  mi ha causato tanto stress sul lavoro, avrà qualche risvolto positivo negli hobby di una persona qualsiasi, o no?

Passo a descrivere la soluzione, vediamo dopo se ho centrato questi obiettivi…

Io ho lavorato con Ubuntu Jaunty, ma tutto quello che ho fatto dovrebbe essere fattibile su qualsiasi linux con poche modifiche e anche su windows o mac con qualche modifica in più (in teoria!).

Per prima cosa, conviene provare se i caratteri elfici sono già funzionanti sul proprio sistema (tentar non nuoce, ma non ci sperate): apriamo un qualsiasi browser (che è già aperto se leggete questa pagina) e dirigiamoci sulla pagina di Prova delle Tengwar Unicode. Se le prime due righe sono in elfico, siete a cavallo, altrimenti vedrete dei quadrati o altri simboli e questo vuol dire che vi manca il font giusto.

Io ho scaricato il font gratuito Code2001 da Tengwar Unicode Fonts e l’ho subito installato nel mio sistema:

• copiato il file CODE2001.TTF in /usr/local/share/fonts/ (ma anche ~/.fonts può andare)
• lanciato il comando sudo fc-cache -fv per aggiornare la cache dei font

A questo punto è opportuno ricaricare la pagina di test (eventualmente dopo aver chiuso e riaperto il browser) per vedere che i caratteri appaiono correttamente!

Bene, il sistema a questo punto permette di visualizzare e utilizzare le tegwar… ma come si può fare?

Tabella caratteri aperta sulle tengwar

Lanciando la Tabella caratteri (Menu Applicazioni – Accessori), i nuovi caratteri possono essere trovati in:

• Area Uso Privato, a partire dal carattere U+E000, oppure
• Area-A a uso privato supplementare, a partire dal carattere U+F2000.

Sono gli stessi caratteri ripetuti: sul mio portatile preferisco la seconda area, perché nella prima alcuni codici sono utilizzati anche da altri simboli e al momento di utilizzarli devo comunque selezionare il font Code2001. Dalla figura si vede che questo font contiene anche le rune de Il signore degli anelli.

Se, come me, state usando Gnome, allora è possibile usare la seguente combinazione di caratteri:

CTRL+MAIUSC+U, codice esadecimale, INVIO

È sufficiente ricordarsi a memoria il codice corrispondente al simbolo desiderato… io che sono troppo vecchio per queste cose (è una citazione?) mi sono fatto un foglio di calcolo con i codici. PDF dei codici unicode delle tengwar

GEdit per scrivere elfico in passwd

Una volta imparato, si possono scrivere questi caratteri dovunque, anche in un file di sistema come /etc/passwd.

Nome in elfico nel pannello

Avendo fatto come nella figura, il mio nome in elfico appare sia nella finestra di login di GDM sia nel pannello in alto a destra.

Applet Tavolozza caratteri elfici

Per aiutarmi nella scrittura, mi sono configurato nel pannello anche l’applet Tavolozza caratteri, come in figura, anche se è sicuramente migliorabile (per ora ho messo alcune vocali elfiche e la e con la dieresi che serve per scrivere Lastë).

A questo punto il limite è l’immaginazione.:)

All’inizio dell’articolo esponevo quali potessero essere i motivi per tutto questo lavoro che ho fatto… beh, posso dire che sono abbastanza soddisfatto.

E’ necessario installare almeno un font che implementi i caratteri elfici: credo che non si possa fare a meno di questo… così come si devono installare i font adatti per le scritture orientali.

Sfruttando la codifica Unicode, si possono scrivere tutti i caratteri (latini, elfici, giapponesi, russi, ecc…) in qualsiasi posto, anche un semplice file di testo, dato non c’è bisogno di formattere il testo, cosa che invece era necessaria prima di Unicode.

Un problema può essere il cambio di codici quando le tengwar verranno accettate definitivamente in Unicode (ma non mi sembra ci sia fretta, la proposta risale al 1997). Fino ad allora c’è il rischio che quelli che io vedo come caratteri elfici, qualcun’altro veda associati a simboli differenti: in effetti le aree in cui sono attualmente inseriti sono denominate per “uso privato”!

A questo punto cosa manca? Sicuramente sento la mancanza delle “modalità automatiche” di scrittura elfica (quenya, sindarin, inglese, italiano…) che sono una caratteristica dei programmi esistenti di scrittura con le tengwar, ma come ho detto precedentemente questi usano i normali caratteri latini che diventano caratteri elfici grazie a vari font. Forse bastano pochi interventi di programmazione per adattare un programma esistente.

I wrote this a while ago purely to find out whether any of the fonts on my PC had a glyph for the Unicode “Sound Recording Copyright” symbol (℗).

You can search by typing in either of the top boxes, searching by the name of the character in the top left box or going directly by unicode character point in the top right box.

You can download Font Finder here.

Have a play, and let me know if you have any problems with it.
It’s a .net 3.5 app, so you’ll need to get that if you haven’t already got it.

With thanks to Michael Kaplan for providing useful input over on his blog: http://blogs.msdn.com/michkap/archive/2008/07/01/8674304.aspx

អ្នកណាហ៊ានថាមិនអាចប្រើភាសាខ្មែរក្នុង

Apple iMac, iPod, iPhone

ទៀតគឺខុស ខុស ខុស

Xenotypeលែងជាដំណោះស្រាយតែមួយគត់ទៀតហើយ  ដើម្បីមើលភាសាខ្មែរលើប្រព័ន្ធ ដំណើរការApple Mac OS។

លោកហុងមកពីKhmer OS បានស្រាវជ្រាវឃើញវិធីអាចអោយយើង ប្រើអក្សរខ្មែរយូនីកូដ ក្នុងកម្មវិធី Microsoft Office 2008, FireFox, Safari, Adobe’s Products (Indesign) ជាដើម ក្នុងប្រព័ន្ធដំណើរការ MAC OSបានហើយ។

ប៉ុន្តែយើងមិនទាន់ដឹងថាពុម្ពអក្សរខ្មែរទាំងនោះនឹងអាចដាក់ អោយបងប្អូនប្រើនៅពេលណាទេ ព្រោះពុម្ពអក្សរទាំងនៅស្ថិតក្នុង ដំណាក់កាលសាកល្បងនៅឡើយ។

ចំណែកឯលោកUponastar វិញបានចេញកម្មវិធីកញ្ជប់ភាសាខ្មែរសំរាប់ iPod Touch / iPhone រួមមានទាំងពុម្ពអក្សរខ្មែរយូនីកូដ ក្តារចុច បញ្ជីប្រើប្រាស់ ជាភាសាជាដើម។

សូមទៅកាន់គេហទំព័រ http://www.iphonekhmer.com

Pics’ Source: iphonekhmer.com

ទាយយកកម្មវិធី​Khmer Unicode MacOSX Lite V2.o ពី http://nokorsoft.wordpress.com/2009/11/13/khmer-unicode-mac/

ដូចនេះទោះបីជាមិនបាន១០០%  យើងអាចនៅតែអាចនិយាយបានថា

យើងអាចប្រើខ្មែរយូនីកូដលើប្រព័ន្ធ Apple Mac OS X និង iPhone OS បានហើយ។

Mr. Hong from Khmer OS has found a way to make Khmer OS Unicode fonts display correctly with software runing under Mac OS X such as MS Office 2008, Firefox, Safari, Adobe InDesing; however, the font development process is still in testing stage. We do not know when  and how the fonts will be released.

On the other hand, Mr. Uponastar released Khmer Localization Package including Fonts, Keyboard, Khmer Menu, Khmer Contact, Weather and Theme for iPod Touch & iPod Phone. Please visit http://www.iphonekhmer.com for more details.

Please download Khmer Unicode MacOSX Lite Version 2.o  from http://nokorsoft.wordpress.com/2009/11/13/khmer-unicode-mac/

So all together, Yes we can use Khmer Unicode fonts on Apple Mac OS X & iPhone OS . It is not a dream anymore.

Okami's Lamy fountain pens

I

One of the marks of being a good prof is when they see that students didn’t understand something and go over it again. My algebra prof has made a habit of this, especially when he goes over stuff in class too quickly. This happened last week and left me quite furious. Someone must have mentioned something to the prof², and he went over what we saw in the last hour again. This took an hour out of the three-hour class. I appreciated. I realized that there were some undergrads in our class, who weren’t familiar with some of the more abstract concepts of category theory³. It was an issue. Class was great today. I was drinking my strong milk tea and noting stuff down. We saw direct sums and (co)universal objects. Having proofs done with commutative diagrams is so elegant and simple.

Limit of a diagram is a cone, via Wikipedia

II

I’ve got a small term paper due in December. I’ve decided to do it on the Clifford algebras of cubic forms. It’s supposed to be quite short, but I’ll pump out quite a bit of stuff so that I can nail an A.

The algebra prof told us that we’d have a take-home midterm exam. I haven’t mentioned to too many people that I have all of the solutions to all of the Hungerford problems¹.Well, most of them. I noticed today while I was printing them out that there were a few sections missing. I was definitely feeling good after he explained everything again. Not that I didn’t understand it beforehand, but it made a difference that he cared enough to go through it once more, at normal speed.

III

Canadian Multilingual Standard keyboard layout, via Wikipedia

I always use a Canadian Multilingual keyboard. For some reason, I’ve been blindly hitting a combination of keys that would give me the multiplication sign ‘×’. I was trying to figure out how this happened when I found this page on Wikipedia. I’ve kind of become interested in the Unicode characters which are available on standard keyboards. My favorites include the superscript numbers I use for footnotes, † → the dagger symbol, ‡ → the double dagger symbol, and this star symbol → ★. Other favorites include the curly brackets → { } and the abreviation for number → №. There’s the degree symbol → ° as well as the tilde → ~ and the horizontal bar → ―.

IV

I think that we’ve completed our intro into category theory. I’ll have to learn the rest by myself, as we’re delving into the structure of groups now. I’m looking forward to that, as well as Galois theory and commutative ring theory.

I really like how some proofs are just completed by commutative diagrams. It’s just so elegant. I don’t know why I find these algebraic topics so interesting. They are elegant and abstract. In analysis, we are seeing the Lebesgue measure and integral. The prof keeps saying ‘mearable’ instead of saying measurable. He also says ‘lebesgue’ by pronouncing the ’s’, which is silent, as any French-speaker will tell you.

Another weird thing was the word integer. I was convinced that it was pronounced integer with a hard /g/, not a soft /j/. This happens because I speak three languages fluently. My wife’s favorites include catholic and patio. When I’m tired, I put the emphasis on the wrong part of the word. She finds it cute.

This is how you pronounce: ĭn’tĭ-jər.

Follow me on Twitter @djrange

* * * * *

[¹]: Algebra, by Thomas Hungerford, Springer Verlag, GTM Vol 73.
[²]: Not about me, but about the fact that he went too fast and that they didn’t understand. Towards the end of the class, I mentioned out loud that I didn’t get most of what he had been explaining because he was going too fast. I was basically just mumbling in my beard.
[³]: The yes girls‡ are undergrads. I wasn’t surprised to find this out. For some reason, the prof didn’t want to get into a too high level of abstraction.

[†]: I thought this was a good play on words.
[‡]: see this footnote in this post.

Computers internally work on numbers. This means that characters need to be coded as numbers. A typical arrangement is to use numbers from 0 to 255, because this range fits into a basic unit of data storage and transfer, called a (8-bit) byte or octet.

A character code is something that defines how that numbers corresponds to characters. Most of the chracters have the same assignment for numbers 0 to 127, used for character that appear in English as well as in many other languages: the letters a-z plus their uppercase equivalents, the digits 0-9 and a few punctuation marks. Many of the code numbers in in this so-called ASCII set of characters are used for various technical purpose. For French texts, for example, you need additional characters such as accented letters. These can be provided by using code numbers in the range 128-256 in addition to the ASCII range., and this give room for letters used in most other Western European languages as well.

Thus you can use a single character code, called Latin 1, even for a text containing a mixture of English, French, Spanish and German, because these language all use the Latin characters with relatively few additions. However you quickely run out of numbers if you try to cover too many languages within 256 characters. For this reason, different character codes were developed. For example, Latin 1 is for Western Europian Languages, Latin 2 for several languages spoken in Central and Eastern Europe, and additional character codes exists for Greek, Cyrillic, Arabic etc. Character codes that use only the code numbers from 0-255 are called 8-bit codes, since such code numbers can be represented using 8 bits. Things change when you need to combine languages in one document and the languages are fundamentally different in their use of characters.

As you know the numbers of characters needed for Chinese and Japanese is very large. They just would not fit into a set with only 256 characters. Therefore, different strategies are used. For example,2 byte (octects) instead of one might be used for some character. On the other hand the character codes developed for the needs of East Asian Languages do not contain all the characters used in the world.

The solution to such problems and many other problems in the world of growing information exchange, is the introduction of a character codes that gives every character of every anguage a unique number. This number does not depend upon the language used in the text, the font used to display, the character, the software, the operating system ar the device. It is universal and kept unchanged. The range of possible number is set sufficiently high to cover the current and future needs of all languages.

The solution is called Unicode, and it gives anyone the opportunity to say, “I want this character displayed and the number is….“, and have herself understood by all systems that support Unicode. This does not always guarantee a success in displaying the character, due to lack of suitable font, but such technical problems are manageable. Much widely used softwares including Windows, Mac OS X, and Linux, has supported Unicode for years. However, to use Unicode , all the relevant components must be “Unicode Enabled”.

Resources for Further Study :

http://www.unicode.org/

http://en.wikipedia.org/wiki/Unicode

-Abhishek Singh

We need to define a column that will contain externally provided identifiers. Their length will vary from seven to ten characters, all English letters, both upper and lower case mixed with numeric digits, with an even distribution of lengths.

Which of these data types will be more efficient?

By John Arnott

Answer: char(10)

Explanation: A varchar (or nvarchar) column carries two bytes of overhead to hold the actual length of the data. Since the average size of the data is 8.5 characters — (7+8+9+10) / 4 –, a varchar data type would take an average of 10.5 bytes for each entry. A column defined as char(10) needs just 10 bytes. As the column will hold only English language letters and digits, we don’t need the additional bytes required by the national character data types. They use two bytes for each character, so would take 20 bytes for nchar or 19 bytes for nvarchar (average lenghth 8.5 doubled for national characters, then add two bytes to hold the length).

» Discuss this question and answer on the forums

You want to display a unicode code point as the char it actually represent ?

For example, display for U+00CE the Î character it represent. (here is the list of all romanian special characters)

Quite strange, I didn’t find a ready to work code for that instantly (as I usually do)

For our task, we will need the berlioz’s unicode2utf8 (that support 4 bytes utf, initialy I got a function that supported only 3 bytes and got errors, if you need 6 bytes support, see the Unicode_to_UTF function) function.

Here is the trick, unicode2utf8 requires as argument an integers, 00CE in our example (and the unicode notation) is hex, everything we need to do is to apply the hexdec function.

Php code:

echo unicode2utf8(hexdec("00CE")); // Result: Î

// Or the function that will recognize U+ in front of string, and will skip it to show the character
function unicodeCodePointToChar($str) {
	if (substr($str,0,2) != "U+") return $str;
	$str = substr($str,2); // Skip U+
	return unicode_to_utf8(array(hexdec($str)));
}
echo unicodeCodePointToChar("U+00CE"); // Result: Î

Why I would ever need that, you will ask, well I need to implement sphinx’s charset_table convert logic on the user’s string. Here’s the map I used, kindly provided by someone on a pastebin.

So if a user search for “bălan”, he will actually find both “bălan” and “balan”.

There are many resources available that you can read and that you will assist in the migration from a old Delphi version to Delphi 2009/2010 (Unicode).

• Unicode Migration Statistics Tool (This utility will hopefully assist you in collecting useful statistics on how hard (or not) it would be to migrate your older applications to Unicode.)
• Delphi 2009 and Unicode
• Delphi 2009 strings explained by example
• Upgrading a major project to Delphi 2009
• Delphi and Unicode
• Dr. Bob Delphi 2009 Unicode
• Delphi 2009 – Unicode in Type Libraries
• On Strings and Unicode in Delphi 2009
• Delphi in a Unicode World Part I: What is Unicode, Why do you need it, and How do you work with it in Delphi?
• Delphi in a Unicode World Part II: New RTL Features and Classes to Support Unicode
• Delphi in a Unicode World Part III: Unicodifying Your Code
• CodeRage 4 : Using Unicode and Other Encodings in your Programs