Herman Bos open source guerilla

Ambition readjustment

Thanks for the comments earlier regarding the client communication. It definitely was a help. After another evaluation we spoke through our design with W&W for a little advice. Which got us to realize we don't have the resources to build the management framework to the ideal end result we had in mind.

This doesn't mean it is a lost cause. We just have to leave out some stuff and focus on the core functionality first. Although we have some pretty good ideas for the client communication now we will leave it as is for now (no useful code) and get the backend designed and implemented. This way we can get the functional part ready before the end of our current development window and at least implement the functionality to automate some tasks. It won't be hard to implement the other features as the time flows on. We can just check one or more different kinds of client communication on top of it later. For the master < -> slave communication we should have something working without too much pain since we don't have to bother much about network communication (we'll use perspective broker (twisted) or pyro (which I discovered today)).

I think we can produce something useful within two months or else a pretty good idea how it should not be done. Key point is we'll have development model with a more evolving nature and extend its functionality during time instead of doing it all at once.

More updates will follow.

Client communication problematics

We are making a more technical design now for our management framework but it seems it won't work out as we had initially planned. We were thinking about using XML-RPC as a client protocol: it is simple and has many available libraries, so it should be quick to implement a client and to integrate it in other software.

The problem is ofcourse that we want more then xml-rpc seems to offer. Especially the data we want to return with an xml-rpc request is much more extensive then supported. With xml-rpc you can only return one result but work around that limitation by using arrays and structs and nesting them. However we want to include so much more information that this would become mad.

We want to include information in a response from which the client can build up an interface which can include tables, forms, buttons, text and maybe a tree structure. Besides that the client should know which data is editable and know which data can be submitted to functions on the server side.
Basicly this comes down to such an extensive XML document that we might as well make it a web interface after all. However this would screw up the integrate with other software goal and you cannot make a nice client for the platform of your choice.

I would welcome any thoughts about this matter.

Management framework

For the last month I'm working together with Kim Chee on a management framework for OSSO (my company). Actually I'm not sure if we should call it a "management" framework, its more of an "system administration" framework. Its probably most clear when I start with the reason why we wanted to build this "management" framework: We deploy customized IT environments based on opensource software (feel the daily administration part coming?).

It works as a beauty but the on-site administrator doesn't have the knowledge to effectively "manage" it. Its not that he couldn't do it, its just not easy, error prone and basicly its not what he is trained for. Another aspect what we wanted to tackle was the time consumingness of the daily administration, filling out a form is simply so much faster then modifying three different configuration files on three different servers.

We started off with looking at the available open source projects, not much luck. Most administration tools are service and/or server oriented (webmin-like) which take away the need for a console, but don't remove the need to know what your doing. Besides not solving the problem they tend to have the nature to inflict mental pain and are even more time consuming then working in the console.

There are also some tools which would work but are only useful for one specific task (like managing users in the LDAP). Then there are a few tools which were quite modular but to achieve what we had in mind it would become a rewrite. So you end up writing your own.

What we want

We want a task oriented system (thats what we tend to call it anyway). The end-user selects a task, if necessary searches for an object, adds/modifies data and is done. A real world example: thinclients that automaticly login. To add or modify a thinclient we need to change DNS (reverse and forward), DHCP and LTSP (which are on three different servers). Now the local admin adds a new thinclient to the network: Chooses add thinclient, fills in a name, description, mac-address and optionally an IP and done.

How do we have that in mind?

A framework which consists of a master server and an optional amount of slave servers. A slave server has modules loaded which correspond with the services running on that same machine. These modules simply implement their little part (like modifying BIND9 configuration files and `rndc reload`). On the master server you can write "tasks" which handle the incoming request from the client and call the modules on the slave servers to get the job done.

Too make it interface independant the client will communicate XML-RPC wise with the master server. This also creates the possibility to integrate it with other software easily.

We will develop the framework in python and we are planning to use twisteds perspective broker for the master-slave communication. We also want to implement logging and notification features.

I like to think about it as a sort of Django for system administration. Its not click and install, you have to fit it too your needs. For example you will have to program these "tasks" in python and if there is no module available you might have to write a module for managing a service. But at the same time there should be an interesting api to support you and you don't have to worry about the underlying work being done.

You could make it do want we do, you could also integrate it in your website and let the framework do the work of registering new users, sms you about it and call your mother with asterisk to tell her the exciting news or some really nifty stuff. We have about 20 pages of text about how we want it with a lot of information and ideas which don't fit this blogpost. I will probably blog about it more when we are making progress with the implementation (in about a month or so). However if you have any questions or your having a bright moment feel free to comment or e-mail.

management framework

A few weeks ago I promised I would blog more about the development of our management framework. Ofcourse many things happened and I didn't blog at all. We finished up the functional design part and now we'll start with the technical design.

Kim Chee Leong, my partner for this project also blogged some stuff inspirated by his recent hatred of UML tools. You can check it out here: http://www.leong.nl/?p=7, he also has some pretty pictures on his blog if you dislike UML.

The main reason why I wrote this blog post: to link to Kim Chee his site. He told me he didn't have any comment spam, so I hope the people who give me 300-600 comments a day will also follow that great link and spam Kim Chee so much he will look for an effective way to prevent the comment spam. After which I can easily adapt the same method.

So this post didn't tell you anything more about the management framework we are building. I don't want to make any promises but it is highly likely I will explain it in further detail.