Monday, November 29, 2010

Your Japanese Name for iOS, part 7

Next I'll need to come up with a dataset of all the segments and waypoints. For reusability, it would probably be best to have a vector representation of each segment. Seems likely that such a dataset doesn't exist yet, easiest way to get one would be to just draw one in some vector editor that lets me display a raster image in the background for reference.

Another option is just to make each segment in Gimp at retina display native resolution. It would be easier, but then porting this to some other resolution like to the iPad would be harder. Of course even for older iPhone I would need a half-res version, although for that I could just scale without redoing anything.

Hmm.. I'm already a bit familiar with Blender. I could make the characters there and get out a triangle list. This solution appeals to me, because I get to play with Blender :-) What would the output look like? I would have one file for each segment. A file that tells where the waypoints are in each segment. Then a file that describes how these segments relate to each character. The most natural way to describe this would be in JSON, and luckily there seems to exist an actively developed JSON library for Objective-C.

segments = {
0 : { // each segment has integer ID
"tris" : [
[0, 2], [1, 2], [1, 3], // coordinates for one triangle
[1, 2], [1, 2], [2, 3],
[0, 2], [4, 2], [1, 3], // each segment would consist of several triangles
"waypoints" : [
[1,2], // exactly two waypoints per segment
... more segments

characters = {
"ka" : [ // each character would consist of several segments laid over each other, order here is significant for drawing
"segment" : 0, // identifier of segment
"offset" : [1,2], // XY coordinates where the segment should appear
"segment" : 1,
"offset" : [2,3],
... // each character consists of multiple segments
"ki" : [ ... ],

It would be pretty straightforward to write a Python script that generates these. Probably less straightforward dealing with Objective-C types when reading them in. Alternatively could make that Python script just generate Objective-C so the types would already be correct and access easier.

Stuff I need:
- Install Blender and get it to work with my keyboard or buy a new keyboard that has numeric keypad
- Exporting triangles from Blender
- Figure out a way to comfortably choose waypoints (repetitive task)
- Also need some tool for composing characters from multiple segments

Uhh am I about to embark on creating some vector drawing program here? x_x Just as a fun thought, if these tools were really good, could I somehow get this task crowdsourced for all Japanese characters? There are thousands of them.

Came up with another option. Use SVG. There's an SVG editor in Javascript already available. It even knows how to overlay vectors on top of an image. Ooh, it has text support. So I don't even need images as I can draw on top of unicode text. For each character in katakana, go to SVG-editor and create a 1280 x 960 image with the desired character at position 400, 800 in size 900. Draw each segment (note: not stroke, a segment) on top of the character. Use as many segments as seems necessary to capture the desired path. Then, place rectangles on top of the complete character to get waypoints. Place them in the order the character should be drawn in. Use different color for each segment's rectangles so that they can be associated with each other. Save SVG (copy & paste) into a file like "a.svg".

This should have all the info in it necessary for revealing the character as it is swiped by finger on the phone.

Have to be more careful in the transitions between segments. I can see a gap between first two segments of A here.

Realized something. I should definitely make these based on our calligraphy videos instead of on an existing font, because then the videos and the vector art would match exactly, making it possible to do some cool transitions between them in the future (for what purpose I'm not sure). Also it would clarify the issue on copyright, as I wouldn't be basing the art on any existing font.

So I need the last frame of each of these videos in order to do this. Sounds like a job for ffmpeg.
ffmpeg -vframes 1 -i frametest.m4v -f image2 output.png

This gets the first frame. But how to get the last one?