A road named NoNameUno

In a recent discussion about a problem, I came to know an intersting problem. The problem is really an old one, namely, document managment. You may say what is there to solve?. Just have large house or office or warehouse, have some high powered printer/copier, ask warehouser or some other companies to cut more trees and make papers to keep your records ( i.e. the documents). Well, not exactly what we were discussing. As you know an old information is no information. And structuring, organizing, analyzing, persenting ... are all very important in every phase of life.

Electronic document processing encompasses email, IM, conference, meeting materials, fax, images, videos etc., etc. Servers who handles these in a distributed fashion are really not that efficients. They often becomes bottleneck for highly loaded document transactions. Approach to solve these are (a) put more hardware/software to distribute the loads (b) improve the efficiencies of the components that made up your document management systems. Solution (a) is not that interesting since it would cause a managing nightmare, it will have multiple points of failures, it will cost more money, it will consume more power (yeah, talk about green ...)

So following the option (b), the first thing to look at is the network processing power. Since 10GB is not yet a household thing, the attention goes directly to protocol processing. When it comes down to Windows server, we are locked with a bunch of LSPs in user mode, and protocol stacks (including NDIS warppers). Most of us know that latency-wise UDP is faster than TCP but it does not have any reliablity. Moreover, if sockets are used ( which happes to be a predominant API for user level network programming in an IP environment), a lot of background processing is left to the work-horse of network implementations of the underlying OS. We all know that most of the general purpose OSes are not real-time, so there is nothing to blame about those network stack implementations. They have their purpose in life, and they solve certain sets of problems.

The obvious questions are ---

1) Can we play unfair?

If we can, then a kernel mode compnent can do some efficiencies.

2) How hard it is to come up with a bare bone protocol implementations that would bypass the TCP stack?

3) Can a kernel socket library help improve the situation?

You might say, firstly all of them have their places and there are tradeoffs. Secondly you might question, How do you know where are the problems?

Answer to second one is easy, and I will, as usual, let you wonder ...

For the first one, there is nothing to answer, life is a tradeoff !!!.