UPDATE: Mandala screenshots (below)

I’ve begun encoding “Robot Dreams”, a short story about a robot named Elvex (LVX-1) whose positronic brain has been uniquely imprinted with fractal patterns, and as a result has learned how to dream.  This text also features Susan Calvin, the mother of robot psychology in the continuity of most of Asimov’s robot stories.  In my encoding of this text, I’ve run into several challenges:

• I’m finding “otherness” more difficult to determine than I’d expected.  This story in particular is challenging, because Elvex has become more “human-like” due to the unique architecture of his brain– a fact that appalls his creator and Susan Calvin.  The more Elvex describes his dreams, appearing increasingly “human”, the more the human characters try to distance themselves from him and emphasize his robotic characteristics.  In this situation, there is a definite tension between “other” and “same”; I can’t ignore that tension by making that attribute “null”, but how can I determine otherness in such an ambivalent circumstance?  …One solution is to look at the source’s motivation.  Is the source saying/doing something to create distance between human and robot, or to draw them closer together?  This raises a new challenge:
• Can a reference then have multiple sources?  Can multiple sources have different motivations, and thus represent different levels on “otherness”?  If the answer is yes, how do I encode this?  …The answer I’ve come up with is to nest my pr_ref tags.  It’s still too early to tell if this is an effective strategy, but I’m trialing it.
• How do I define my type attributes when it seems that the reference is fulfilling more than one of the possible types?  (e.g. in “Robot Dreams”  Susan Calvin interviews Elvex in her characteristically cold, clinical way.  Most of her questions/statements directed at Elvex can be construed both as “interactive”– since she is “interacting” with the robot– and “descriptive”– since she is describing the robot.)  One possible answer is to look at the possibility of multiple sources again.  The other is to identify a hierarchy of types: emotion trumps interaction trumps description, since all references “describe” something, but not all references “describe” an interaction, and not all interactions are emotional.  Without clearly setting this rule out, I think this is a strategy I followed when encoding “Someday”.  When there is a clearly a situation of multiple sources looking at motivation can again be valuable, and using nested tagging seems the natural answer.

I chose “Robot Dreams” because it has several elements that I felt needed to be explored in my analysis of Asimov’s robot stories.  First of all, whereas in “Someday” the two human characters were male children, in “Robot Dreams” the two human characters are female adults.  I wanted to see if gender and age played a factor (note: my tweaked encoding currently doesn’t catalog age as a factor– if it looks like this might be valuable information to mine, I may add it in future iterations).  Secondly it included Susan Calvin.  Although I have not, as yet, developed an element structure to analyze principal human characters, it has always been my intention for Calvin to be my first attempt.  Not only is her name synonymous with Asimov’s robot stories as a recurring character, but she plays a unique role in them as a foil for the various robots she psycho-analyzes; it would be a valuable exercise to compare the relationship references to her with those of the principle robot characters in the same stories.  Is Susan Calvin characterized as more robot (“other”) or more human?  In comparison, are the robot characters more or less human? Does she elicit more of an emotional response from the figures that interact with her?  An examination of reference sources in this analysis is useful too: does she express emotion more or less than the average robot?

Finally, the problem of “otherness” is central to this text.  I feel that the tension between being “too human” and “too different” is one that makes Asimov’s work so universally engaging, and has not been explored to its fullest.  My XML encoding can– hopefully– reveal exactly how that tension is expressed through the relationships in the text.

***

I have completed a first encoding of the principal robot references in “Robot Dreams”.  Here are screenshots of Mandala evaluating “otherness” from the perspective of the three characters: Elvex (principal robot), Susan (principal human), and Linda (secondary human).  Click on the thumbnails below to view the images in full size.

So you have got hold of your video files, now how the hell do you turn them into a DVD that you can watch on your DVD player, Xbox or another PC? Keep reading for the solution…

I won’t go into how to get hold of your movie files as these could come from handycams, mobile phones, torrents (not that i’m suggesting you illegally download them) or your own DVD rips. However you get hold of your digital movie files you have the issue of actually watching them. Do you suffer the smaller computer display(okay I know some people have 37 inch displays etc) or turn them into a DVD to watch on your large HDTV that you bought for that very reason?

Well I’m guessing you would go with the TV. I’m also assuming you don’t want to stream over wifi to an xbox or ps3? That would be a whole other story which I’ll address later on.

The software I would suggest as it has always worked well for me is DVD Flick

DVD Flick is FREE! And it is nice and straightforward to use. It has loads of features but keeps it’s easy for begginners too. Full info can be found here , this explains all the file formats it supports.

All you need to do is find the file you want to burn to DVD, decide on menu options, if you want it burnt to disc after encoding or an iso for burning later, and then click Go.

As it’s free give it a go and see what you think, I would not suggest running it on a low power processor though, as it will take till the end of time to finish.

Enjoy.

Xbox 360 multimedia capabilities are very good, with streaming functions available not only for Windows systems (see here for streaming movies using Ubuntu).

There is a caveat though, the lack of subtitles functionality for movies, and the only way to use them is to hard-sub (encoding the movie again applying subtitles on it).

Several tools are available for every operating system, but the easiest one I use is mencoder (part of the MPlayer project), a command line tool for encoding audio and video files.

Mencoder is available in almost every Linux distribution, for install on Ubuntu just type:

sudo apt-get install mencoder

I encode each movie in Xvid format (which Xbox is capable of play), with this command:

mencoder mymovie.avi -ovc xvid -oac mp3lame -xvidencopts fixed_quant=1 -sub mymovie.srt -o mymovie-encoded.avi

where obviously mymovie.avi is the original movie without subtitles, mymovie.srt is the subtitles file and mymovie-encoded.avi is the resultant output file.

The output file will be larger than the original one because I used the fixed_quant=1 option, that outputs larger files but with best quality (you can use 2 or 3 but the quality downgrades).

Encoding may be very slow on older systems, but once finished it can played as any other media…enjoy!

Here is the link to the Connect Session I filled in for Premiere CS4 for beginners.

A very quick runthrough on some of the features that people new to Premiere may not be aware of.

Artigo bacana, que explica como evitar o uso do encodeURIComponent usando o PHP para trocar variáveis com o Flash via GET.

Link para o POST

Via Blog do Gabriel Ricci

Após penar muito, achei uma solução de encoding de variaveis no PHP para uso no Flash. Se você precisa preparar strings com caracteres especiais (como acentos) para usa-las no Flash via GET, você deve usar encodar a váriavel já encodada em UTF8 com o rawurldecode, dessa maneira:


function flash_decode($str){
return rawurlencode(utf8_encode($str));
}


Dessa forma, você evita ter que usar a função encodeURIComponent no JS, que só está disponível no JavaScript 1.5.

Amy's looking for something good to watch

I don’t know about you, but I find myself watching more and more TV on my computer.  As I’ve mentioned in a previous blog, I’m a big fan of Glee.  However, I am usually not home when it airs.  And even if I were, I couldn’t watch it because we don’t get FOX.  (We don’t have cable or satellite TV, and FOX’s over-the-air signal doesn’t get to us, for some reason.)  In order to get my Glee fix, I have to watch it online.

Last week, CSI did a trilogy that started on CSI: Miami, continued on CSI: NY, and concluded on CSI.  I was intrigued and wanted to check it out.  But I knew I wasn’t going to be home to see all of them.  So what did I do?  I watched them online.

My husband also wanted to watch these shows, and watching them on one of our laptops is far from ideal.  Fortunately, we are able to connect a laptop to our television, allowing us to watch the online episodes on our TV.  It’s not a perfect situation, but it beats watching them on a 15″ laptop screen.

And in the past year, online video quality has improved.  Last week, YouTube announced that it would have videos in full HD 1080p.  A lot of the networks already post their shows in HD.

On my way home last night, I heard a story on NPR about watching TV on your computer on your TV.  They talked about how easy it is to hook up your computer to your television, and they also talked about some products that can be a permanent addition to your entertainment system.  It was good timing, because I knew I’d be watching the final installment of the CSI trilogy later in the evening.

But it also got me thinking about the increasing importance of the quality of online videos.  As more and more people watch videos online and as more and more people watch those online videos on their televisions, the videos need to look good.  That starts with the format used in the shooting of the video and ends with how the video is encoded for the web.  That sounds very technical, I know, and it can be easily overwhelming.  When clients come to Tweedee Productions, we start by asking them what the video is going to be used for, how it’s going to be used, and where it’s going to be viewed, and with that information, we can make sure the finished product will look great, no matter where it’s viewed.

I have been asked numerous times about how to encode purchased DVDs into the video equivalent of MP3. For me, and many others, this is MP4.

The below steps will explain how to convert a DVD into a MP4 format, complete with metadata and artwork. (The process is nearly identical on Windows and Mac – Sorry Linux fans, I have yet to dabble with this on that platform, but I’m sure it’s almost the same too)

Even though you own the DVD, the act of ripping it and converting to another format is still very much a legal grey area. (I hope this guide doesn’t get pulled because of this!)

You will need a few applications;
MacTheRipper (Mac) or DVDShrink (Windows)
Handbrake (Mac & Windows)
MetaX (Mac & Windows) or Subler (Mac)

And a few sites to help with obtaining the metadata;
IMDb (General Data)
Netflix (Genre and Outline)
Barnes & Noble (Chapters)
tagChimp (General Data and Chapters)
Wikipedia (General Data)
Get Video Artwork (High resolution artwork)
Blockbuster (UK Ratings)
Internet Movie Poster Database (Last resort if you cannot find decent artwork)

Ripping the DVD

Handbrake can convert straight form the physical disk itself, but I prefer to rip it to my Hard Drive first.

With MacTheRipper, it’s easier to insert the disk and click “Go!”. This will rip the entire contents of the DVD, allowing you to pick out the parts you need later. Ripping a specific Title can be a bit fiddly.

DVDShrink can do the same thing via the “Entire Disk” button. But it is much easier to rip just the Title of the Main Feature using the Re-Author option.

To save getting into unnecessary detail, for the purpose of this guide, rip the whole Disk.

Encoding the Movie

1. Launch Handbrake. You should be prompted for the Source File(s). If not, select File>Open.

2. Handbrake will scan the Contents. Once finished, open the Preset tray and select Universal.

3. In the main screen pick the Title you want to encode. This is usually the longest one for the main feature. Play the DVD elsewhere and take note of the Title number to be sure.

4. Type in the location and name of your desired output file (for the Windows version, all special characters must be removed. You can change the filename after).

The default setup here should be fine for most people, so it is safe to skip onto the Audio and Subtitles Step – you will be left with a perfectly acceptable video file. If you’re interested in some of the finer points, read on…

5. In the Video tab click Picture Settings.

5a. Set Anamorphic to Strict – this will force Handbrake to keep a pixel-for-pixel resolution of the original (no scaling like with the loose option)

Because of this, you have to adjust the Cropping to keep the video the correct shape;

5b. Set the crop to Custom and adjust the numbers so that they are multiples of 16 (this is to do with macroblocks – there are numerous post online explaining this, but for now trust me, 16 is the magic number) and even numbers (e.g. if you find you have 71 and 73 for the top and bottom, you should change them to 72 and 72 – this is to do with efficient encoding).

Keep note of how much of the video you are cutting away or adding back. adding too much back may leave more black edges than you would have liked. Cutting too much away may take away too much of the film. The last one is highly unlikely as, when overscan on CRT TVs was an issue, the extreme edges of the film were cut off anyway.

5c. If your source is interlaced or Telecined (again, plenty of posts about this elsewhere) select the relevant Filters. Decomb can be left on if you are unsure, as this is like a ’safe’ de-interlacer that will only be brought into effect if needed.

5d. Now check your Source, Output and Anamorphic numbers;

The Output should be the source minus your crop values, exactly. The Anamorphic should with the same height as the Output. This can be ignored if you have chosen “Loose Anamorphic”. I prefer strict as you are left with an exact pixel-for-pixel copy of the original without any scaling.

6. Onto the Audio and Subtitles Tab.

6a. The default here is for track 1 to be an AAC Pro Logic II mix-down of the 5.1 AC3 source, with Tack 2 being a passthru of the source. This is fine in nearly every case.

The two tracks are for compatibility. The Apple TV and Media Players can pick off the AC3 5.1 track (Amplifier friendly) whereas the iPhone and iPods can use the Pro Logic II track, as it’s just stereo. The ProLogic II allows those with PLII amps to playback the pseudo-5.1 from the portable devices too.

Ideally you would do away with both of these and choose the 6-Channel discrete AAC, as this is the ideal equivalent to AC3 5.1. But due to a slow uptake on this standard we are forced to use the 2-Track method above, breaking the MP4 standards. Plex and XBMC can down-sample 6-Channel discrete AAC to AC3 5.1, but they are the only ones, leaving you with a file un-playable on the iPods and iPhone and a mix-down to something non-AC3 from the Apple TV. A real shame, but what can you do?

You can set the 2nd Track to None if the source is only Stereo or Dolby Surround (no point have two sets of 2-Channel tracks).

6b. Choose any subtitles you would like hard-encoded to the movie file. These will be ‘burnt’ onto the video itself so be sure before you choose any (such as Forces Subtitles where some Movies may have translated scenes). I will post another guide for adding soft-subs to your files once I have found a bullet-proof method.

7. The Chapters Tab.

7a. Fill in the chapters via whichever method suites you (copy and pasting from one of the sites listed above, manually from the DVD box, or choose your own by watching the movie).

The Advanced Tab can be left alone, as this is really reserved for the hard-core tinkeres.

8. Click Add To Queue or Start depending on how many DVDs you are encoding. Depending on your Computer, the encoding process can take anywhere from 1 hour to 1 day (yes my 8 year old PC used to take a whole day to do this!).

Adding the Metadata, or ‘tags’

If you’re only interested in a file to playback the Movie, indifferent to cataloguing your collection, you can skip adding any data to the file. I, however, like having all the information stored within the file, like with MP3s (it’s only Apple Media players and various Operating Systems that can read this data for now, but this will change).

Open your weapon of choice, be it MetaX, Subler or one of the more obscure ones and populate all the required fields. The sites listed above are great for this.

Once complete, Save/Write the data to the file.

You’re done!

The file you are left with is about 33% to 50% the size of the DVD Title and can be imported to iTunes with all the data intact. It will be exactly the same resolution as the original (if you choose the ’strict’ method) and can be played back on almost any device – one file the rule them all!

I will post a guide for Blu-ray and High Def rips once that process has been more streamlined and standardised.

Some likely questions and things to note

Why choose the ‘Universal’ Preset?
This is the most compatible format to use. It leaves out all the unnecessary ‘extras’ that make it un-playable on some devices. One example is the B-frames added for the Apple TV preset – this only saves about 10% more file space but makes no improvements to video quality.
The consoles are always changing their requirements and therefore rendering their presets useless too often. They are being removed from the next release I believe.
The AVI container is a dying format and will also be removed from the next release of Handbrake.
The device-specific presets reduce the resolution for playback via the device’s screen only – too low for TV use really.

Be careful of blank chapters at the end of a Title.
Some DVDs are authored with a chapter at the end of the movie no more than about 3 seconds long. This is to help with the menu navigation (I think) and can be left out (select the previous Chapter as the last one in Handbrake’s main screen). leaving it in will result in an unwanted entry at the end of your Chapter List – not a huge deal, but irritating non the less.

Double-check your Movie duration once encoding is complete.
Handbrake will add or remove a few frames here and there to keep things in sync (nothing that can be spotted during playback, don’t worry) so the output file’s duration should be the same as the original, give or take a few seconds at the most. The movie studios are very aware that programs like Handbrake can circumvent most attempts to prevent encoding, so have started adding deliberate errors to the DVDs. These errors are handled by stand-alone players no problem, and skipped over, but many can cause Handbrake to choke. I am suffering from this issue with Iron Man – selecting the subtitle track for the bits of translation at various scenes causes Handbrake to drop whole scenes from the movie, leaving me with a file a full 15 minutes too short. This is because they haven’t kept that particular subtitle track’s time-stamp in sync with the main movie… sneaky cretins.

You can set the Constant Quality to anything from 59% to 62%.
Anything lower will result in noticeable artefacts. Any higher and you are wasting space. In this version of Handbrake, the percentage is in relation to the original film source, not the DVD – setting it to 80% will result in a file the same size as the original DVD, any higher and the size balloons. In comparison, a Blu-ray is considered 95% percent of the original film source.

The same method can be followed for TV Shows and 4:3 sources.
Just select the required Title, or Chapter range, that relates to the episode you’re wanting.

Some DVDs will flat-out refuse to rip.
For these you can plump out for DVDFab.

Please post any questions relating to this guide in the comments below.
I will respond as best I can.

Objective:
To obtain mp3 encoding capability in VirtualDub on a Windows 7 64-Bit system.

Programs:
VirtualDub 1.9.7
LAME mp3 Encoder

Guide:
Step 1: Unzip all the files in VirtualDub to its own folder.
Step 2: Right-Click on LameACM.inf and click ‘Install’

Restart VirtualDub, if you still aren’t able to access LAME mp3 encoding please continue…

Step 3: Go to Start and search for and run ‘regedit’
Step 4: Go to the registry path:
HKEY_LOCAL_MACHINE>SOFTWARE>Microsoft>Windows NT>CurrentVersion>Drivers32
Step 5: Change the entry labled: ‘msacm.l3acm’
From “C:\WINDOWS\system32\l3codeca.acm”, to “C:\WINDOWS\system32\l3codecp.acm”

Restart VirtualDub. If that didn’t work. I apologize this solution isn’t for you.

There’s always a lot of talk about ripping DVDs as a backup, but not a lot of chatter about ripping DVDs onto mobile platforms.

Not that it’s hard for me to do this stuff, since I was trained for it, but to the regular Joe, they either buy a DVD and or they download the movie from iTunes and have to pay twice.

I can’t be the only one who finds this ridiculous, right? And what about discs that have French and English titles married together in the VOB files – an extra pain in the ass for nerds?

Well, these are things that float in my brain…;)

I just tried doing that and realized that lame doesn’t support globbing. That’s where bash comes in:

for f in *.wav; do lame --preset extreme "${f}"; done

God I love unix.

Joel Spolsky nel 2003 scrive  – The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) -

Gioia e dolore di ogni informatico.

Perchè gioia? Prima dei sistemi di encoding, fra i quali Unicode è un po’ il principe, regnava l’ASCII “chaos”, nell’articolo è spiegato molto bene, e come sappiamo, la standardizzazione è il punto di partenza per creare insieme qualcosa di grande.

Perchè dolore? Bisogna provare per credere! Non avete idea di quanti siti web scritti ante-2000 sono scritti in un encoding, hanno un meta tag che indica quello sbagliato e magari l’http server ne usa un altro. C’è sempre da impazzire un po’ con queste cose: un po’ è perchè sono implicitamente complesse, un po’ perchè molta più gente di quanto pensiate, definendosi sviluppatore o peggio ‘informatico’ (?), pavoneggia la sua ignoranza parlando di charsets e affini. “Ah ma tanto io faccio applicazioni plain text: non mi interesano queste cose!”. Ecco, se un americano può sembrare solo ignorante a dire questo, un italiano è anche peggio… perchè?

Gli accenti maledizione! Quante volte avete visto strani punti interrogativi o quadratini su siti internet, e-mail o documenti vari? Ebbene, quello è un problema di encoding. E se vi dicessi che i cinesi hanno svariate centinaia di lettere nel loro linguaggio? e badate bene, qui non si parla ancora di internazionalizzazione del software, quello è uno step molto successivo.

Io consiglio sempre questo articolo di Joel on software per iniziare a farsi un’idea di questo mondo incredibile.

Here’s the link to my latest Connect Session – Using CS4 to encode to multiple formats such as H.264, FLASH and BluRay
http://pacific.adobe.acrobat.com/p61913948/
Cheers

Verify MySQL Client and Server Using utf8

* On old server
mysql> show variables like ‘char%’;
+————————–+—————————-+
| Variable_name            | Value                      |
+————————–+—————————-+
| character_set_client     | latin1                     |
| character_set_connection | latin1                     |
| character_set_database   | latin1                     |
| character_set_filesystem | binary                     |
| character_set_results    | latin1                     |
| character_set_server     | latin1                     |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+————————–+—————————-+
8 rows in set (0.01 sec)
* On new server
mysql> show variables like ‘char%’;
+————————–+—————————-+
| Variable_name            | Value                      |
+————————–+—————————-+
| character_set_client     | utf8                       |
| character_set_connection | utf8                       |
| character_set_database   | utf8                       |
| character_set_filesystem | binary                     |
| character_set_results    | utf8                       |
| character_set_server     | utf8                       |
| character_set_system     | utf8                       |
| character_sets_dir       | /usr/share/mysql/charsets/ |
+————————–+—————————-+
8 rows in set (0.00 sec)

*Sample my.cnf file to get above results
[mysqld]
init_connect=’SET collation_connection = utf8_general_ci’
init_connect=’SET NAMES utf8′
default-character-set=utf8
character-set-server=’utf8′
collation-server=’utf8_general_ci’

[client]

default-character-set=utf8

Make sure Apache using UTF-8 as default character set

* Edit httpd.conf
”’AddDefaultCharset UTF-8”’

Make sure PHP uses utf-8

*Edit php.ini
”’default_charset = “utf-8″”’

Convert your Latin-1 collated tables to UTF-8

Make a backup

Since this step is the first place we’re actually going to be changing data within the database, the very first thing we’ll want to do here is make a backup of the database. Note that since in MySQL Character Set Hell, a mysqldump of the database is likely to have syntax errors in it, I recommend making a backup of the raw database files. In order for this backup to have clean data in it, you’ll need to shut down your mysql daemon while making the backup. If you’re server is a RedHat- or CentOS-like operating system using vendor-supplied RPMs for the MySQL packages (and default paths in your my.cnf), the following set of commands ought to do this for you:

# service mysql stop
# cd /var/lib
# tar zcvf mysql-backup.tar.gz mysql
# service mysql start
You’ll want to make sure your database server has enough space under /var/lib on it to store the above backup, of course.

Convert those tables

Please note that if your data set does not contain a mix of single- and double-encoded UTF-8 characters in a Latin-1 character set (ie. you either have one or the other) then there are safer conversion techniques than what I’m about to document below. (Specifically, see this page: http://dev.mysql.com/doc/refman/5.1/en/charset-conversion.html ). This step assumes that your data set contains tables with a mix of single- and double- (and triple- and quadruple-) encoded UTF-8 characters in a single Latin-1 collated table.

The procedure for converting your tables to UTF-8 collation will vary depending on which sources you’ll read, but the way that worked for me was to do a carefully-crafted mysqldump, alter the dump slightly, then re-import the dump back over the database itself. To minimize the chances of clobbering data in tables that are already properly UTF-8 collated, this really only needs to be done on tables using the Latin-1 collation to begin with. Do discover which tables these are, you can run:

mysql> show table status where collation like ‘%latin1%’ \G
*************************** 1. row ***************************
Name: UserInfo
Engine: InnoDB
Version: 10
Row_format: Compact
Rows: 2
Avg_row_length: 8192
Data_length: 16384
Max_data_length: 0
Index_length: 65536
Data_free: 0
Auto_increment: 3
Create_time: 2008-06-01 22:48:11
Update_time: NULL
Check_time: NULL
Collation: latin1_swedish_ci
Checksum: NULL
Create_options:
Comment: InnoDB free: 0 kB
9 rows in set (0.00 sec)

mysql>

(Yes, “latin1_swedish_ci” is the name MySQL uses for Latin-1 collation.) You’ll need to run the above for all databases on the MySQL server, and compile a list of affected tables. For each of these tables, run:

1. # mysqldump -u dbuser -pdbpass –opt –skip-set-charset –default-character-set=latin1 –skip-extended-insert databasename–tables  tablename > database.table.sql
2. # sed -i.bak -pe ’s/DEFAULT CHARSET=latin1/DEFAULT CHARSET=utf8/’ database.table.sql
3. # cat database.table.sql | mysql -u dbuser -pdbpass databasename

Clean up double-encoded characters in your current data set

Before going any further in these instructions, I highly recommend taking a close look at your application. At this point, everything everywhere should be speaking UTF-8, and all your tables should have the UTF-8 collation. In an ideal world, especially if you started with a relatively clean data set, this may be all you need to do to make everything peachy in your application. So take a moment to look at your application (and specifically content generated from rows with non-ASCII characters in them) and verify you really need to go further in this how-to.

Ok, so from here on out, we’re going to assume that you do still have problems displaying characters as they should be in your application, and that this is due to having double-encoded characters in your UTF-8 collated tables. (Note that if your terminal and mysql command-line client are speaking UTF-8, you can verify the problems lie with the data itself with a few SELECT statements targeted at rows containing the data.)

Caveats

As a few final notes before moving on just to drive the point home:

The remainder of these instructions will not work if you are not speaking UTF-8 everywhere, or if your tables are not using the UTF-8 collation.

* If you have a relatively small number of rows affected by character encoding issues at this point, it may just be simpler and safer to fix these by hand.
* If you have a relatively small number of different types of double-encoded UTF-8 characters it may be simpler to convert these characters directly using techniques detailed by Derek Sivers in his excellent blog article on the same here: http://www.oreillynet.com/lpt/wlg/9022
These instructions assume you won’t have single- and double-encoded UTF-8 characters in the same field of a single row of any given table. This is a relatively safe assumption, unless you’ve already partially-completed manually fixing the data as described by Derek Sivers in the above point. If you do have such fields in your data set, these instructions will definitely corrupt them. (ie. You’re better off either finishing your conversion using Derek’s techniques, or reverting the data back to the state they were in before attempting Derek’s techniques.)

Scope of these instructions
The instructions in this section of this document will detail how to fix double- (and triple- and quadruple-)-encoded character issues for one column in all rows of one table in the database. These steps will need to be repeated for any affected columns in all tables in your database. Scripting this is left as an exercise for the reader (or as an incentive to become a customer of Blue Box Group. )

For the examples I’ll be using below, we’ll use the following example name conventions. Obviously, you’ll need to replace these with the actual names that apply to your database:

Table: mytable
Field / Column: myfield
Type of the above field / column: mytype
Temporary table (name is arbitrarily chosen): temptabl

Find tables with fields containing multi-byte characters in them

At this point, only those rows which have non-ASCII characters in text-containing fields (like CHAR, VARCHAR, TEXT, etc.) should have problems with double-encoded UTF-8 characters in them. The really nice part about UTF-8 here (and tables containing UTF-8 characters) is that any non-ASCII characters are by design multi-byte characters. We can exploit this fact to find tables and fields with rows containing non-ASCII characters.

mysql> select count(*) from mytable where LENGTH(myfield) != CHAR_LENGTH(myfield);
+———-+
| count(*) |
+———-+
|      669 |
+———-+
1 row in set (0.28 sec)

If the above query returns a count of zero, this means that either:

* There are no non-ASCII characters in any rows of the table for this field. Therefore, this field of this table does not need any correction and can be ignored.
* Your table is not using UTF-8 collation (which is bad, because if you’ve been following my instructions thus far it should be).
* Non-ASCII characters in this field in this table have already been munged by previous attempts at correcting character encoding issues and have been turned into single-byte characters (more than likely a question mark). If you have any rows with fields like this, the information needed to automatically convert them back into the multi-byte characters they should be has already been lost, and these rows will need to be fixed by hand.

Anyway, you will want to go through all text-like fields in all tables of your database and find any which contain rows that have multi-byte characters in them like this. These rows can potentially have double-encoded UTF-8 characters in them and will need conversion as detailed below. I suggest compiling these into a list you can automatically parse with a script which will do the remainder of the steps below.

Also note that it is very important at this stage to make note of the field type and any associated parameters
(eg. the length of the VARCHAR, etc.). I suggest taking a close look at the table’s description to get this information.
mysql> describe table;
+———-+————-+——+—–+———+—————-+
| Field    | Type        | Null | Key | Default | Extra          |
+———-+————-+——+—–+———+—————-+
| msno     | char(3)     | YES  |     | NULL    |                |
| title    | varchar(50) | YES  |     | NULL    |                |
| subtitle | text        | YES  |     | NULL    |                |
| author   | varchar(30) | YES  |     | NULL    |                |
| pubdate  | date        | YES  |     | NULL    |                |
| series   | varchar(50) | YES  |     | NULL    |                |
| journal  | varchar(50) | YES  |     | NULL    |                |
| arttext  | text        | YES  |     | NULL    |                |
| id       | smallint(6) | NO   | PRI | NULL    | auto_increment |
+———-+————-+——+—–+———+—————-+
9 rows in set (0.00 sec)

mysql> select count(*) from table where LENGTH(myfield) != CHAR_LENGTH(myfield);
+———-+
| count(*) |
+———-+
|       63 |
+———-+
1 row in set (0.01 sec)

Convert double-encoded UTF-8 characters to proper UTF-8 characters

Here’s where I stopped following the instructions in the link above. I couldn’t get the commands listed to work with my version of MySQL so I used phpMyAdmin.

1. Using phpMyAdmin convert the table collation to latin_swedish_ci
2. Change the type from whatever type (ie. text) it is to BLOB
3. Change the table’s type back to whatever it was (ie. text) and the collation to utf8_general_ci

Let me explain what the above does:

* The first alter table command tells mysql to convert the UTF-8 encoded characters into Latin-1. At first, this appears to be the exact opposite of what we were trying to accomplish above by making everything everywhere speak UTF-8. But please remember that by “double-encoded” UTF-8 characters, I really mean properly-encoded UTF-8 characters that were run through MySQL’s Latin-1 to UTF-8 conversion algorithm one too many times. By converting the UTF-8 collated field to Latin-1, this effectively tells MySQL to run those characters through the reverse algorithm: Convert UTF-8 encoded characters to Latin-1. This has the effect of taking those double-encoded UTF-8 characters and turning them into single-encoded or properly-encoded UTF-8 characters.

So what about those rows which already had properly-encoded UTF-8 characters in them? Well, that’s what some of those warnings were. But we’ll get to that in the next step sub-part.
* The second alter table command just converts the textual field into a BLOB (or raw binary) field. This does no change to the actual binary data underneath and no conversion algorithm is run.
* The third alter table command takes that BLOB field and converts it back to a textual field with the UTF-8 encoding. Since MySQL has no algorithms defined for character conversion when going from a BLOB to any other data type, these last two steps effectively allow us to fool MySQL into converting the field from Latin-1 to UTF-8 collation, but skip the character set conversion algorithm that would otherwise re-double-encode our now pristine properly-encoded UTF-8 characters. Pretty sneaky, eh.

Whilst trolling around iusethis.com I came across this great process viewer utility called atMonitor.  This goes beyond what Activity Monitor does and allows you to see in a more coherent and user-happy way the processes running on your computer such as applications, daemons and so forth, their CPU usage and RAM usage.  The best part of it all is the ability to set priority of a process.

When using ZippyRippy you may notice the encoding of the ripped file can take some time depending on what else you’re doing on your machine.  To this end I have often found it useful to set the priority of a process so that other processes don’t hog the CPU providing I really don’t need it for a while.  Here’s how we can do this with ZippyRippy.

We need to find what ZippyRippy is running to determine what processes need some boosting.  ZippyRippy has two main binaries ARffmpeg and ARmencoder.  ARmencoder does the ripping and ARffmpeg does the encoding.  Once they are running they can be selected and the priority changed to allow a boost in encoding. Here’s how we change the priority temporarily.

Click the button in the toolbar labeled “Renice”.  Not quite sure what that means but if you hover the mouse over that button it reveals a tooltip that states you can change the priority of a process.

By clicking that you reveal a drop down sheet that shows a slider which allows you to change the priority of the process.  Keep in mind that changing the priority of a process may slow down other applications, I’ve found that upping the priority of ARmencoder or ARffmpeg is better utilized if you do not intend to use your machine during the encode or rip.  Move the slider to the highest setting or wherever you feel comfortable and click “Set”.  This will make the process a high priority while it’s running, if it is quit and relaunches the priority will be set to whatever default it is given by the system.  If you click “Set and Remember” then the process will always have the priority you set here whenever it launches.

*NOTE: Please read the atMonitor documentation to lean more about how it works and what it does if you are unsure of the process or setting priorities and their effect on the system.

Also while looking at some of the commands in the Terminal, I came across “renice” and “nice”. These are commands built into OS X which alter the scheduling of processes.

From the Terminal man pages:

I shoot a lot of stuff in windowless rooms of swish hotels, pressed into the service of Presentation Theatres. It’s a living. But just last week, whilst shooting in a very new venue next to St Pauls Cathedral, London, I was confused with a problem.

The lighting in some rooms appeared to strobe or flicker. In a subtle manner, not the big horrible flicker of bad Fluorescent or Sodium Vapour lighting. It was so subtle that I reckon many mortals would not have noticed it. I’ve tried to capture it even though the compression process has lessened its effect:

Well, at least you can see it in the viewfinder, rather than when viewing rushes back at home AFTER you can do anything about it. And if you can’t see the ripple effect in the initial shots, don’t worry. It’s just the Vimeo/YouTube compression. It IS very visible on the raw footage.

You’re not seeing things, it is there.

So in these situations, it seems that you can’t stop it, you can only lessen it. Slow the shutter down or turn it off. Experiment with the shutter – you might find a happy harmonic which lessens the effect. At least the compression process seems to minimise it.

한글 utf-8 vim 저장 & 읽기 테스트

제대로 읽고 제대로 쓰기 위해선 3가지 옵션이 맞아야 한다.
encoding : 파일 저장이나 읽어오는 방식과는 상관없고, 화면에 보이는 것과 상관있다.
           이 옵션은 터미널 설정에 따라 영향을 받을 수 있다.
fileencoding : 파일에 저장되는 코드, encoding와 fileencoding이 같으면 상관없고, 다른 경우 읽거나 쓸 때 변환 된다.
fileencodings : 여러 캐릭터셋을 적어주면 파일을 읽을 때 하나씩 체크해서 맞는 캐릭터셋으로 자동으로 fileencoding을 설정해 준다.

* 현재 이 문서는 gnome-terminal(터미널 인코딩 설정 utf-8)에서 vim 을 사용해서 작성됐다.
  vim 인코딩 옵션은 encoding=utf-8, fileencoding=utf-8, fileencodings=utf-8,euc-kr 로 설정되어 있다.

* gnome-terminal 인코딩 설정과 vim encoding 설정이 맞아야지만 제대로 보인다.

En la versión actual de JDeveloper (11.1.1.1.0), las aplicaciones por defecto están codificadas en “windows-1252”. Codificación que nos dará problemas cuando tengamos textos con acentos en nuestra base de datos.

Afortunadamente esto se puede cambiar en cada una de las páginas, mediante la propiedad contentType.

<jsp:directive.page contentType="text/html;charset=utf-8"/>

O también podemos modificar las preferencias por defecto del IDE, para que de esta forma todas las aplicaciones que desarrollemos hagan uso de UTF-8 o de la codificación que queramos. Tal como explica didier.laurent en su blog.

Pues esta tontería es lo que me dio problemas, el otro día, durante varias horas.

Según informa didier.laurent en su blog, en la próxima versión de JDeveloper tendremos UTF-8 como charset por defecto.

Converging lines of evidence indicate that stress either before or after learning influences memory. Surprisingly little is known about how memory is affected when people learn while they are stressed. Here, we examined the impact of learning under stress in 48 healthy young men and women. Participants were exposed to stress (socially evaluated cold pressor test) or a control condition while they learned emotional words and neutral words that were either conceptually associated with or unrelated to the stressor. Memory was assessed in free recall and recognition tests 24 hours after learning. Learning under stress reduced both free recall and recognition performance, irrespective of the emotionality and the stress context relatedness of the words. While the effect of stress was comparable in men and women, women outperformed men in the free recall test. These findings show a memory impairing effect of learning under stress in humans and challenge some assumptions of current theories about the impact of stress around the time of learning on memory formation.

from Neurobiology of Learning and Memory