On the way to happy computing

Многие слышали о технологиях которые используют Google, Microsoft, Yahoo у себя внутри.
В основном, это благодаря количеству усилий, которые они тратят на популяризацию себя.
Многие другие популярные ресурсы также имеют свои технологии, разработанные in-house, некоторые со временем выпускают их в мир(как например LiveJournal’s Memcached)
Одним из таких интересных ресурсов является Amazon.com. Недавно я встретил интервью с Werner Vogels [...]

Majority of us knows that databases are read-optimized products and most of them are not optimized for writes.
In this post I would like to describe what’s going on under the hood of mysql that can help us to find origins of some well-known rules.
So, let’s start with MySQL MyIsam table:
- MyIsam tables caches only key [...]

The article at codeproject describes framework that uses client-side Silverpoint CLR to implement web-based volunteer grid computing. Not a cool one, but the idea’s interesting.
Taking into account dramatic grouth of AJAX popularity and its adoption by a community, the paradigm shift towards grid-computing-in-the-browser is quite possible.

It’s a cool platform for distributed computing and research.
Just check out the numbers:
approximately 4,000 processors, three terabytes of memory, 1.5 petabytes of disks, peak performance of more than 27 trillion calculations per second (27 teraflop).
-Source

On September 19, 2007 Heb Sutter did a presentation on a subject at MS.
Snippet from description:
High-level languages insulate the programmer from the machine. That’s a wonderful thing — except when it obscures the answers to the fundamental questions of “What does the program do?” and “How much does it cost?”
The C++/C#/Java programmer is less insulated [...]

Almost everyone posted news on Ubuntu Gutsy. New enhanced desktop, flash support in firefox and etc.
I decided to give it a try, and… spent a lot of time playing with cube, visual effects and so on. Yeah, it’s fancy but eventually I’ll switch it off and get back to work ;). IMHO the most useful [...]

24 и 25 сентября в Москве прошла конференция посвященная высоконагрузочным системам.
Часть докладов уже доступна на RuTube. Не буду дублировать все доклады, приведу лишь те, что успел просмотреть и мне понравились:

Anatolix рассказал о трудностях с которыми сталкивались в Яндексе при борьбе с высокими нагрузками.
http://rutube.ru/tracks/197396.html?v=377843ff9b0f9eec261b9b8378e238d7
Большинство проблем в принципе классические в Information Retrieval, +/- специфика архитектуры [...]

MySql comes with extremely useful feature full-text search. The minor thing that bored me was the minimum word length that mysql can index to. It was 4 letters by default on my machine.
To change the value to desired one you need to:

set ft_min_word_len variable up in my.ini file
restart mysql
ensure that the change [...]

 
To realize the value of ONE YEAR,
ask a student who failed a grade
To realize the value of ONE MONTH,
ask a mother who gave birth to a premature baby.
To realize the value of ONE WEEK,
ask the editor of a weekly newspaper.
To realize the value of ONE HOUR,
ask the lovers who are waiting to meet
To [...]

http://channel9.msdn.com/rss.aspx?ShowID=13&format=mp3
Dot Net Rocks
Hanselminutes
Dr Neil’s Notes
Fly With Me (Joe D’On - pilot in the US)
Fly With Me X (extra bits)
Betty In The Sky With A Suitcase (flight attendant)
Inside the Net (I like the interviews with web 2 startups - Leo Laporte and Amber Macarthur)
Mark Kermode’s film reviews (from Five Live and Simon Mayo at the BBC)
Milahseconds
Mondays
New Scientist [...]

Today I meet a strange one working with ec2 using command line tools.
Each request could not be executes and I received each time Unknown host: ‘https://ec2.amazonaws.com’
Nslookup gave me:
Non-authoritative answer:
Name: ec2.amazonaws.com
Address: 72.21.210.54
And finally, traceroute died with “Request timed out” message.
After some digging I found a workaround
export EC2_URL=https://72.21.210.54
 Now it works perfectly fine

For hierarchical clustering have a look at this resource
Browsing the web I met interesting open source framework for search clustering engines.
here is their table of algos (Original source)
 

.algorithms, .algorithms td
{
border:1px solid silver;
}

Algorithm
Author
Speed s*
Hierarchical
clustering
Other features
Example results

100
200
400

FuzzyAnts
Steven Schockaert
2.17
8.70
16.93
yes

london

HAOG-STC
Karol GoЕ’embniak
0.04
0.11
0.28
yes

london

Lingo**
Stanislaw Osinski
0.34
0.52
0.84
no
multilingual clustering
london

Rough k-means
Ngo Chi Lang
1.38
6.76
27.73
no

london

STC
Oren Zamir
(impl: Dawid Weiss)
0.04
0.10
0.23
no

london

Lingo3G***
StanisЕ’aw OsiЕ”ski
(Carrot Search)
0.03
0.06
0.13
yes
multilingual clustering, synonyms, advanced tuning, scalability (5000 snippets in [...]

In most Java VM implementations, the garbage collector has the right to stop a thread in order to reclaim memory no longer used by that thread. Such garbage-collection pauses are the main reason threads running in a regular JVM cannot guarantee the exact timing of their execution…
 
Original source - http://www.artima.com/lejava/articles/javaone_2007_realtime_gc.html
mp3 podcast is also available there.

Accidently met this resource surfing the web.
http://www.google.com/patents
A quite useful one, seems to me.
E.g. you can have a look at such stuff as:

System and hardware module for incremental real time garbage collection and memory management
Packet processing system including a policy engine having a classification unit
Type checking in java computing environments

Java C++ proxy objects
System and methodology [...]

It looks like I blog a lot on google :). But today I found an interesting thing.
For instance:

you can find buffer overflows , one more example
or backdoors

Simple, doesn’t it? Yes, it simplifies a code inspection a lot…

Browsing the web met an interesting resource - Peter Norvig’s aritcle (Peter is Research Director at Googe).
It looks like he gives obvious directions, sometimes funny, but formed in a good way. Yeah, 10 years is a time frame that makes developer a mature one…
Snippets:

“the maximal level of performance for individuals in a given domain [...]

Discussion of the assembly language cost of exception handling on the x86 Windows and x64 Windows platform

Herb Sutter - Discussions of concurrency & C++ futures September 2006 meeting of the Northwest C++ Users Group.

Scott Meyers @ NWCPP: Red Code/Green Code - Generalizing Const
C++ compilers allow non-const code to call const code, [...]

Do you want to know how Big Boys(like Digg, Google, YouTube, Flickr) look like inside? In this case the resource I found recently will be interesting for you.
Let’s look under the hood:

YouTube Architecture
Flickr Architecture
LiveJournal Architecture
Amazon Architecture
Google Architecture
GoogleTalk Architecture
FeedBurner Architecture
MySpace Architecture
Wikimedia architecture

source - http://highscalability.com/

For a last couple of weeks I’ve been involved in a huge C++ project that takes donkeys year to compile.
What would you do during this compilation? Exactly, switch to another task. It’s enough to be easily carried away, and it’s very wasteful to check a build each 5 minutes.
So, I decided to compose a notification [...]

Skype is not just a VoIP tool, it’s a black box with the multiple protection layers.
In addition Skype’s an interesting look at software architecture in general.
Look - you know, everyone is acquainted with classic client-server architecture, it’s easy to implement and maintain. But usually it’s accompanied with the expensive hosting infrastructure(do you know the [...]

I found very interesting site, and the guy who leads it is also keen on distributed and scalable stuff. So, if you have a time,
just make a break and read a couple of enjoyable posts on scalability and certainly browse the library at http://www.royans.net/arch/
 
OK, I think I’ll write a couple of posts related to [...]

The following article contains interesting thoughts on skype internal routines.
http://www.heise-security.co.uk/articles/82481/0
 
References

STUN - Simple Traversal of User Datagram Protocol (UDP). Through Network Address Translators (NATs) - http://www.ietf.org/rfc/rfc3489.txt
Traversal Using Relay NAT (TURN) - http://www.jdrosen.net/midcom_turn.html

Hi here! I’m keen of programming and technology and am about to share my thoughts from time to time here. That’s it for now, have a good one!