|
Powered by
|
|
Section: All | News & Politics | Geek Stuff | Devel | Non-existent Life | Random | Food! | Life |
Mon, April 21, 2003
BlogMatcher optimization
BlogMatcher: I'm re-working the match-finding portion of the system right now. The PHP-based script worked remarkably well, but with 2600 blogs and counting, it was starting to show its limits. The database approach failed miserably (I'll spend more time on that later), so the new plan is to convert chunks of my PHP code to C/C++. I just finished the C version, which does the searches in roughly a third of the time (i.e. 1 second vs 3 seconds), but I realized that one of the slowest processes could be further optimized if I use C++/STL. While taking a dump, I also realized that maybe I could run multiple processes to do the search since it's fairly disk intensive and I'm sure the programs waste quite a few CPU cycles waiting for disk I/O.
Ultimately, my goal is to meet Google standards (show results in less than 1 second). On a semi-related note, I still want to work for Google.
I'll post more in a couple of hours...
|
|
Ryo Chijiiwa
I'm a biologically Japanese, culturally American, Germany-raised, socially liberal, politically independent, gun-totin', code writin' dude. My life is currently sponsored by Google.
|