SimonHF's Blog

Just another site

What is SXE? October 9, 2010

Filed under: Uncategorized — simonhf @ 10:05 pm

Since I’ve blogged a few times about its performance, several people have asked: What is SXE?

What is SXE? In a nutshell, SXE is an easy way to build robust & scalable network programs without comprising run-time performance or memory usage.

How does SXE achieve this?

1. By providing a software development environment for the fastest language, C, where rapid development (not traditionally associated with the C language) can occur.

How does the environment enable rapid development?

a. Traditionally tedious to create and maintain header files and makefiles are generated automatically meaning that the developer need only concentrate on creating .c files. Creating a new library is as easy as creating a new folder with a new .c file.

b. Release, debug, and code coverage builds happen automatically. SXE itself is developed using the same mechanism and is protected by 99.9% code coverage. This means SXE is rock solid code and programs developed using the environment will also be rock solid. This means bugs are always minimal and the latest github version is always a release candidate.

c. The necessity of using a traditional debugger to slowly single step through code has been eliminated by the use of extensive code instrumentation techniques which also double as an efficient code comprehension mechanism.

d. Although not forbidden the traditional use of pointers in C is strongly discouraged — for reasons of reducing complexity — and alternative mechanisms are provided to achieve similar results using simple, human-friendly, and easy to debug integers.

2. By providing a generic layer of network handling code on top of the cross-platform and asynchronous libev. Want to create an HTTP- / SMTP- / DNS- / whatever- server? The common network handling code has already been created for you. All you have to do is write some simple handlers for network connect, read, and close events.

3. By providing powerful and generic data structures to maintain state and chronological information even in a multi-threaded environment. It’s a bit like having an ultra powerful NoSQL solution right inside your program. For example, you have one million peers connected and want to know who has been in the read state the longest? The generic data structure will tell you. Want to know which is the second longest? The data structure will tell you. Want to multiplex between the one million peers and another limited resource? Just use two data structures and request the oldest member in state ‘waiting for resource’ when any member of the other data structure becomes state ‘resource free’. Memory leaks (a traditional problem with C programs) are completely eliminated by using fixed memory structures which effectively mirror the maximum capacity of the server hardware they are running on. The effects of DDOS attacks become a thing of the past because programs are engineered to use a fixed amount of memory and therefore even in an attack the server will never ‘go into SWAP’ or become unstable; as soon as the attack ends then regular functionality resume immediately.

In this way it’s possible to rapidly create robust & scalable network programs with surprisingly few lines of code and without compromising run-time performance or memory usage.


8 Responses to “What is SXE?”

  1. JT Says:

    You refer to this quite a bit in all your articles, but not once reference where this code is or being worked on. Its interesting in theory, but how about allowing some of us to work with it. Basically, link, or it didn’t happen.

    • simonhf Says:

      SXE stands for Sophos Extensible Events and is actively being worked on by Sophos Inc. SXE has been open sourced with the MIT license. There is a version available at github (Where else? :-)) which gets updated from time to time by another SXE collaborator, Jim Belton.

  2. Jim Belton Says:

    My branch on github is:

    Note that I only build/test on 32-bit Ubuntu, though generally the code I’m adding has been merged from a branch that is tested on 32 bit debian, 32 bit RHES and 32 bit Windows (XP)

  3. Roland Jonstone Says:

    Is there a single point of access/community for SXE? Are there many players out there? The interweb is not very helpful (sharing a name with a popular music fest will do that).

    Any projects that you’ve used SXE in that the masses can look at?

    • simonhf Says:

      Roland, at the moment there is only github but a general purpose wiki is in the works. Up until now, SXE has only been used in closed-source, ultra high volume commercial applications. There needs to be a first open source application using it. Not sure what would be the best application to show off the technology. Maybe some kind of chat server? Any ideas?

      • Anon Says:

        I have an idea.
        Try porting I2P to SXE.

      • simonhf Says:

        Thanks for the tip, anon. I’ve never tried the I2P stuff. It’s Java, right? What would you be hoping that a libsxe implementation does better than the current Java implementation? Java normally doesn’t perform that badly if handling just a few dozen connections or less. Where libsxe would shine would be handling thousands or tens of thousands of concurrent connections and/or reducing the memory foot-print. But that level of concurrency isn’t needed for I2P, or? I’m interested in hearing your opinion and expectations. And I’m willing to take a look at I2P 🙂

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s