Matt Knox has very interesting past – he wrote adware. This is one of that interesting tasks when you have to understand OS deeply in order to write piece of code that will transparently turn millions of machines into zombies staying undetected.

So, amusing story to read – Interview with an Adware Author

In this post I would like to describe what’s going on under the hood of mysql that can help us to find origins of some well-known rules.

So, let’s start with MySQL MyIsam table:

- MyIsam tables caches only key data structures in memory and let OS caching mechanism to cope with data. Truly, particularly Linux is good at it at some point(Also MyIsam has read-ahead buffer for data). So, if insert/update query updates key data structure, MyIsam will flush key data on the disk. It slows down a whole process. To avoid this you can tweak mysql’s configuration to delay key writes.

- MyIsam comes with a very simple locking mechanism – it’s its table-level locking. So, when user writes data, another one won’t be able to read any row till the end of transaction. Okay, you can say, if insert time is relatively short, we can wait. So, let us see how MyIsam inserts record into a table: Firstly, it scans a table area and tries to find previously deleted row and re-use its room. If no deleted row was found, it appends data to the end of the table. So, if the space is not as important as execution time, you can speed it up telling database to append data to the end of table immediately and disable scans.

For the bulk data management, it’s good to create a table without indices first and import data. Once the import process is finished you can create indices. If you need to append large amount of data to production machine, you can do hot replacement: prepare such table and send signal to application to use another table instead of old one(the type of signal depends on your application architecture: IPC, record in database with polling on application side,etc)

Snippet from description:

High-level languages insulate the programmer from the machine. That’s a wonderful thing — except when it obscures the answers to the fundamental questions of “What does the program do?” and “How much does it cost?”

The C++/C#/Java programmer is less insulated than most, and still we find that programmers are consistently surprised at what simple code actually does and how expensive it can be — not because of any complexity of a language, but because of being unaware of the complexity of the machine on which the program actually runs.

This talk examines the “real meanings” and “true costs” of the code we write…

Video of the presentation at google video:

You can download presentation here

In addition Skype’s an interesting look at software architecture in general.

Look – you know, everyone is acquainted with classic client-server architecture, it’s easy to implement and maintain. But usually it’s accompanied with the expensive hosting infrastructure(do you know the numbers youtube spends on hosting?!).

But Skype developers (and original developers of Kazaa as well) implemented distributed self-organized peer-to-peer network, P2P. So, everyone might be a host.

Okey, let’s go on.

Skype’s executable is encrypted and it dynamically decrypts itself as it load into memory. The dump flush and analysis is difficult since startup code is cleanded after execution and we get executable that cannot be launched. Integrity checks are executed in random order, basically upon incoming call event. Basically there are no static calls and most important functions are called using dynamically calculated and obfuscated pointer.

Interesting, isn’t it? What do they hide from us?

You can find more information and detailed description by Kris Kasperski here

More links:

• General Skype Analysis
• Skype Trojan – how to use skype network to distribute viruses
• How to use Skype with Softice?
• Skype Reads Your BIOS and Motherboard Serial Number

References

1. STUN – Simple Traversal of User Datagram Protocol (UDP). Through Network Address Translators (NATs) – http://www.ietf.org/rfc/rfc3489.txt
2. Traversal Using Relay NAT (TURN) – http://www.jdrosen.net/midcom_turn.html