|
The oh-so-brilliant me I managed to delete the database that holds all the text for this site yesterday and spent most of the day trying to restore the data. I have a full backup from September, but I still needed to manually upload all the entries since then. Ugh. |
 |
Improved Category Generation After that scare with the script taking too long to re-generate this site, I decided to go inside the code to see what I could do to speed it up. It turns out that there was a huge bottleneck in sorting Japanese text with Perl's Unicode::Collator. A little caching using some hashes did the trick so it didn't have to generate a new sort key everytime it needed to do a comparison. It took more than 55 seconds to generate the site before, I nailed it down to about 30 seconds now. It no longer gives me a timeout error. In addition, with the savings in time, I implemented a little counter underneath each section in the manga listings. It now shows the number of books for each hiragana grouping. Hmm...most of my books fall in the "ka-ko" group. I guess I don't need Moveabletype 3.1 after all and can postpone upgrading until I max out the server again. |
 |
Manga Listings Revised I've just revised the manga category page. I knew that scalability would be an issue, but didn't think I would hit it so fast. Previously, if you wanted to browse the manga archives, it would show you all 144 (as of today) covers. That's defintely not a good thing. It's been changed so that it only starts off with a subset of the Japanese alphabet and you can choose. The "all" viewing mode is still there, but it's no longer default (nor does it have book titles on the screen...put your mouse over the image and it should show you what it is). Implementation wise, it's terribly inefficient, but I'll wait until my hosting service complains. |
 |
Features of 'mo Lunar Scribbles It's been a month into re-starting my online journal, or I guess they're called blogs now. Much has progressed in terms of design, but I can finally say that I'm content with the current layout and features. If there's one thing I'm guilty of, it is that I've spent way to much time these past few weeks with the aesthetics rather than content, but I guess that's part of the fun. As of this writing, I will still update the side bars, both left and right, but I don't think they'll vary in appearance. Here's a feature overview of mo' Lunar Scribbles: 1. Fully Automated Mobile Blogging
0) A cronjob that runs a Perl program to check a specially assigned email box for new messages every 15 minutes. This email box is solely for the use of moblogs. 1) I take a picture with my cell phone and send it to the email address mentioned above. 2) The Perl program downloads the message and feeds it to the modified Python program, mail2entry. 3) The Python program parses out the message, separates image from text and assigns the title and appropriate subject. My modifications to it have it call ImageMagick, a image manipulating program, to make a copy of the image with the dimensions of 200x200 pixels in size, maximum while retaining proportions (so that's usually 200x150 for 4:3 images). It also generates even smaller thumbnails (80x80 max) to serve as the rollover images on the index page. The black and white image with noise that serves as the rollover image is also generated at this time. 4) With all that done, the Python script uploads the new entry to Moveabletype for posting. Upon upload, Moveabletype re-generates all the changed pages so that in the entry, there's a smaller image with a link to the larger image, as well as text for the entry. The index pages are regenerated with the new moblog entry and the appropriate rollover code is generated with the new image (this is actually done with PHP when you load the page, Moveabletype generates the PHP code). 2. Thumbnails of Reviews with Ratings
From there, the index pages load the generated thumbnails of the last XX entries and link them with their respective individual archive pages. When you load up the index pages, PHP is used to count and lay out the entires in a nice grid. For some entries, I use the full-size image (width 170px) for added effect. There's also the "Currently Enjoying" section where I have things I'm currently reading/watching/listening to, but haven't written a review yet. Those items are entries in Moveabletype, but are labeled as "Draft" so they don't show up in the archives. In order to pull them out of the database (Moveabletype usually hides "Draft" entries), I used the SQL plug-in. Other than that, they're identical to published entries and show up the same way. Another feature along with the thumbnails is the watermarked ratings on the thumnail images. Upon generation, the entrybody gets check for a "Ratings X/4" text and it grabs the "X" number. Using ImageMagick, it loads up a little sideways banner with stars (random color) and overlays it on top of the thumbnail. Values can go from 0-4, with 0 getting something more interesting than just a blank banner. 3. Moving and Blending
Using the AniMagic Layer Dreamweaver Extension, I made it so that it would move layers depending on which category you wanted to see filtered on the index page. Each entry is in it's own layer and when it's loaded, they're all stacked on top of each other because the browser hasn't determined the length of each entry. Once it has determined the length, the Javascript will stack them on top of each other as if they were cells in a table (that's why you first see garbage on the screen when you first load the page before everything flattens out). When you choose a category, the Javascript code will hide all the layers that are not in the category and perform the animation for the selected categories to stack up at the top of the screen. Another cosmetic touch is the blending of each day's entries. Each day is denoted by different color. There are 7 colors, and to get the color for a particular date, do a mod 7 on the day of month. Anyhow, a plain and boring way of laying it out is to make the entries look "boxy" with abrupt changes in colors between different days. But given that ImageMagick can do gradients, I decided to make it a smooth transition between colors instead. So, I pre-generated all possible combinations of color transitions (7x7 = 49). In the static index pages (non-moving), the appropriate transitions are generated by Moveabletype. In the moving index pages, they're first generated by Moveabletype, but when you choose a particular category to filter on, it will change the gradient to match the next entry. One last thing, I've made the layers emulate a table as much as possible. Layers usually don't resize according to your browser size, unlike tables, but that would lead to an inconsistent design. So I had the layers resize according to the width of your browser. So hopefully, you won't be able to tell the difference between the layer page (only the index page, and only with browsers IE6+ and Mozilla) and the tables page (the safe index page as well as all the archive pages). 4. Smart Media Listing
Now for the hard part. To do the filtering for a group of alphabets/kana, I had to create my own <MTEntries> tag, which I called MTEntriesJ because Japanese kana was my main reason for doing it. So the tag works like your regular MTEntries tag, except it had a few keywords, "filterstart" "filterend" "notwithin" and "excludeprefix" and "rowwidth." These work only on the titles of the entry and a special "sort" tag I have which I'll explain later. filterstart: The first letter, inclusive, of the grouping (case-insensitive). So for example, if I wanted all the entries that started between letter "A" and "E," but without letter "B," I would do: <MTEntriesJ filterstart="A" filterend="E" excludeprefix="B">blah blah blah...</MTEntriesJ> If I wanted all titles that didn't start with an alphabet letter, I would say: <MTEntriesJ filterstart="A" filterend="Z" notwithin="1"> This goes the same for Japanese kana, so for the か～こ (ka-ko) sequence, I would write: <MTEntriesJ filterstart="か" filterend="こ"> But for those who know Japanese, this poses a problem especially when dealing with Kanji. Kanji usually has multiple readings, so I can't rely on the title alone. So in addition, I have a <sort>blah blah blah</sort> tag that I put in the Extended Entry portion. I wrote a parser to parse the Extended Entry for that tag and use it as a key for sorting. The text in the <sort> tag is what's really being sorted on, regardless of the title. Well, actually no, it does two sorts. It sorts on the text within the sort tag first and then it sorts on the title (I just concatenate the sort string and the title to get the sort key). That way, I can guarantee the position of the sort. This doesn't help much in sorting English words, but is a lifesaver in the Japanese. Now what's up with the excludeprefix tag? Well, I needed something a little more fine-tuned. For my Japanese comic magazines, I wanted them to have their own section called "Magazines." But that posed a little problem. If I gave it a title name using any of the Japanese kana, it would fit in with the other books. If I put a non-kana character in the <sort> field, it would put it in the "others" location. Therefore, I needed to make a special exception in the "others" category not to absorb certain character prefixes. So my fix is that for all my magazine entries, I have a hyphen as the <sort>. In the magazine filtering section, I have "filterstart='-'" and "filterend='-'" And in my "others" section, I have an "excludeprefix='-'" And it all works out. 5. Calendar Icons
The "unique" part comes up because I have a few topics that share the same icon (as of today). For example, "Site Info" "Everyday Life" and "Work" all share the same pencil icon. However, if I wrote on those three categories in a single day, it should show up as three pencils in the calendar. IMO, that doesn't look right. So, my hacked MTEntries takes in a parameter for the max number of unique categories to show, and then another parameter for the grouping. I can specify a grouping of [Site Info, Everyday Life, Work] and what I get back from my MTEntries is just one icon to represent any of those three. 6. Random Email Addresses Not really a layout feature, but still helpful. I've documented how it works already, so I won't mention anymore about it. So in summary, this site is very automatic. It took a while to get everything up and running, but now that it is, the only thing I need to do is follow the instructions on the box: Just add words. A lot of these scripts were done in haste with poor poor software engineering practices of documenting and generalizing code (I do proper software coding at work, so give me a break!). I'm reluctant to put them out on display for that reason. However, if you're interested in what I have, I'll try to help you out with the franken-code that I have. |
Using 
Books, Movies, Manga, Anime, and Video Games all are checked for images. Upon generation, I have a Moveabletype plugin that extracts the first image from the entry (for the categories mentioned, the first image is always the cover). When it extracts this image, it checks to see if there's a thumbnail of it already generated. If not, it generates it.
One of the things I really dislike about web programming is client-side stuff. I despise it because there are too many variables to deal with, especially browser incompatiblity. So I first started out with an old-school tables driven page. But then I had this idea of being able to filter out the entries on the index page, so the moving layers idea was formed.
Let me tell you what a pain in ass this was, especially dealing with Unicode and Perl. The idea is to have a sensible listing of books, DVDs, and CDs rather than just having a long list. First off, I 
Each square in the calendar can hold up to 3 different category icons to show what was written that day. Getting the categories to show up isn't too bad, but I had to write another MTEntries routine to get "unique" category icons, and that was a little trick. |
Moving Entries! Spent the last 6 hours dealing with one of my most hated programming tasks. Browser/Javascript programming. I like server-side programming like perl/php/python because it's a contained environment. I know what server it runs on and I know that the output is always the same. However, you never know what you're going to get with client-side programming since everyone uses a different browser. The sliding effect I have can only been seen with Mozilla (and Netscape 6+) and IE5+, I believe. The webserver automatically redirects you to the page, whether it is the regular or dynamic based on what browser you're using. I just hope it works well enough. I really do like the effect though. |
 |
Randomly Generated Email Addresses In an effort to curb the effects of receiving spam from I've devised a few scripts that will automatically create random email address every-so-often that will redirect to my read email account. The idea is that if a spammer harvests this email address, it (spammers are not worthy of human pronouns) will have a window of X days to spam me before that email alias is gone forever and my web hosts blackholes the message. It won't stop the spam problem, but at least I don't have to see it. Of course, I could make a email form, but that's not too interesting. The code I have below is what I've created for myself. It'll probably require some tweaking if you want to use it since I didn't spend the extra time to make it thoroughly generic, but hopefully it'll give you an idea. There are three components to this, a Perl script that runs on the server, a cron job that calls the Perl script, and a PHP script in the web page so that for every request, it'll get the latest email rather than having to wait for the next re-building of the page. Some prerequisites: even though all my pages end with .html, they're all sent through a PHP processor before being displayed. This is done with an Apache directive ForceType. Your web server also has to have a ".redirect" file available. For my host, I have a web page control panel to create email aliases, but that's just a front end to modifying the ".redirect" file. This script will access the file directly. The Perl script (I call it "genalias.pl"):The PHP script (I have this stored as a MoveableType Template named "Get Random Email Link"):#!/usr/bin/perl # Needed, download from http://www.cpan.org (do a search) use Tie::File; ### Constants # The redirect file for email aliases my $redirectFile = "/home/jupiterstar/.redirect"; # This keeps track of the previously used email addresses # so the cronjob knows what to delete from .redirect my $recordFile = "/home/jupiterstar/.oldaliases"; # Number of old emails that are valid after they've # been replaced by a new one (even though the page will # show the new address only, the previous X number of email # address will still be valid, where X is this variable) # Increasing this number means that old email address # will be valid much longer my $staggeredNum = 1; # This is the email addresses to forward to my $permanentAddress = "xxx@xxx.com"; ### Begin Code # The new email address is a random string of length 12 # domain for this address is whatever your host is my $newAddress = generate_random_string(12); my $newAddressTime = time(); # push new address and time into the $recordfile and # remove the old address tie(@recordArray, 'Tie::File', $recordFile) or die "Error tying $recordFile"; unshift(@recordArray, $newAddress . " " . $newAddressTime); my $oldAddressRecord = ""; if(scalar(@recordArray) - 1 > $staggeredNum) { $oldAddressRecord = pop(@recordArray); } untie(@recordArray); # open up redirect file and add new address, deleting old if needed tie(@redirectArray, 'Tie::File', $redirectFile) or die "Error tying $redirectFile"; if($oldAddressRecord) { $oldAddressRecord =~ /^([^\ ]+).*/; # find first item (the address, not time) my $oldAddress = $1; for(my $i = 0; $i < scalar(@redirectArray); $i++) { if($redirectArray[$i] =~ /$oldAddress/) { splice(@redirectArray, $i, 1); last; } } } push(@redirectArray, $newAddress . " " . $permanentAddress); untie(@redirectArray); # Generate Random String # (lifted from: http://www.codeproject.com/useritems/perl_randomstring.asp) sub generate_random_string { my $stringsize = shift; my @chars = ('a'..'z', 'A'..'Z', 0..9); my $randstring = join '', map{$chars[rand@chars]}@chars[0 .. $stringsize]; return $randstring; } 0 0 1,15 * * /home/jupiterstar/genalias.pl After all is said and done, I put the tag <$MTInclude module="Get Random Email Link"$> in my index template and it'll give me the current email address. |
 |
The Return of Lunar Scribbles Officially, it should be the fourth revival of this journal since it's inception way back when. But in fact, I started on this revision last August and I just stopped working on it. As such, I have a 6 month gap since the last time I entered some serious stuff in here. I'm not going to lie and say I won't do it again, but there's a part of me that feels a bit wasteful since I'm paying $20 bucks a month to have this domain name and hosting, but I have absolutely no intentions of fixing or working on any other part of my site. Work occupies a lot of my time, and when I'm not there, I'm at home studying Japanese. Mostly vocabulary now since I think I'm proficient with the grammar. I know who's doing what to whom, but still have problems with the who, what, and whom. All in good time. Anyhow, that's all for tonight/this morning. Got work later today at 8am. |
|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||
