Learning to Code 15 May 2012
Rewinding my career a ways, I want to weigh in on the Learn to Code debate as a non-programmer who coded. I am a professional programmer now, but my previous career was teaching English in High School (you are not allowed to take that as license to mock my grammar).
Programming and the Profession of Programming are quite different things. Programming is being able to efficiently tell a computer exactly what to do in a repeatable manner. The profession of programming is being able to efficiently convert business requirements into bugs.
As an English teacher I programmed regularly in order to make my life easier. I generated vocabulary quizzes (and grading sheets for them), I created interactive epic poetry (I shit you not) with my classes (those studens really grokked epic poetry thereafter), I wrote hundreds of small scripts to calculate various things (many of which could have been done in excel, but I knew perl, not excel), I turned at least one student onto cypherpunks during a study hall, I built various one-off web applications for teachers, classes, groups, etc. I calculated lots of statistics on student performance, tests, and so on so I could better understand and calibrate things (teachers may not always grade on a curve explicitely, but new teachers always do at least hand-wavily as they don’t have tests and teaching well calibrated yet).
Programming is a tool that let me be more efficient, that allows you to automate boring things, and sometimes opens up options which would otherwise be unavailable. I later left teaching, went into technical writing, and then (back) into the profession of programming full time. As Zed Shaw put it well, “Programming as a profession is only moderately interesting. … You’re much better off using code as your secret weapon in another profession.”. I happen to love programming for itself, so programming as a profession works well for me. Code is ephemeral though, and most folks don’t like to “see your works become replaced by superior ones in a year. unable to run at all in a few more” as _why described. Progamming is an exceptionally powerful tool for accomplishing other things.
Java URL Handlers 14 May 2012
There are two ways to register your own URL scheme handler in Java, but they both kind of suck. The first is to set a system property to a list of packages, and then name subpackages and classes therein just right, the second is to register a handler factory. The handler factory approach seems great, except that you can register one, once, ever – and, oh yeah, Tomcat registers one. Given the complete brokeness of the factory registration the sane way is to align packages and class names perfectly, though this is annoying to do, so like all good programmers, I wrote a library to put a nice facade on the process: URL Scheme Registry .
Using it is about as simple as I could figure out – because URL handlers need to be global (given how URL works) there is a single static method. To register the dinner scheme handler you just do like:
UrlSchemeRegistry.register("dinner", DinnerHandler.class);
Et voila, you can now use dinner://okonomiyaki and an instance of DinnerHandler, which must implement URLStreamHandler, will be used if you retrieve the resource.
Note that you can only register a given scheme once, and your handler must have a no-arg constructor.
Under the covers the library adds a particular package to the needed system property for the package/class lookup method and creates a subclass of the class you pass it, using the super convenient CGLib. This way it can leave the handler factory setting alone (so you can use it in Tomcat or others that register the url handler factory), and don’t need to manually set system properties.
You can fetch it via Maven ( search for current version ) in two forms. The first has a normal dependency on cglib:
<dependency>
<groupId>org.skife.url</groupId>
<artifactId>url-scheme-registry</artifactId>
<version>0.0.1</version>
</dependency>
The second puts cglib into a new namespace and bundles it in case there are still cases of colliding cglib versions in the same namespace out there:
<dependency>
<groupId>org.skife.url</groupId>
<artifactId>url-scheme-registry</artifactId>
<version>0.0.1</version>
<classifier>nodep</classifier>
</dependency>
Have fun!
Hello Pummel 26 Mar 2012
I’ve been doing some capacity modeling at $work recently and found myself needing to do “find the concurrency limit at which the 99th percentile stays below 100 milliseconds” type thing. So I wrote a tool, pummel to do it for me.
$ pummel limit --labels ./urls.txt
clients tp99.0 mean reqs/sec
1 2.00 1.03 967.59
2 2.00 1.04 1932.37
4 3.00 1.43 2799.16
8 16.00 3.64 2199.13
16 130.00 7.81 2049.02
8 17.00 3.54 2262.44
12 73.00 5.62 2135.61
16 129.00 7.58 2110.93
12 71.00 5.57 2155.99
14 117.97 6.58 2127.79
12 71.00 5.57 2155.99
$
By default it is looking for that “tp99 < 100ms” threshold, which in this case it found at 12 requests in flight at the same time.
Also really useful is the step command, which just increases concurrent load and reports on times:
$ pummel step --limit 20 --max 5000 ./urls.txt
1 2.00 1.05 956.21
2 3.00 1.08 1854.26
3 4.00 1.24 2415.46
4 3.00 1.41 2836.48
5 6.00 2.05 2444.27
6 9.00 2.54 2358.31
7 11.00 2.92 2398.74
8 16.00 3.38 2364.49
9 23.99 3.99 2257.22
10 35.99 4.43 2258.05
11 54.99 5.10 2157.37
12 72.98 5.94 2020.61
13 87.97 5.94 2187.89
14 125.99 6.40 2187.64
15 125.00 6.85 2188.50
16 130.00 7.39 2163.68
17 134.00 7.86 2163.13
18 143.98 8.35 2156.93
19 156.00 8.92 2129.52
$
Assuming you put this output in data.csv this also plots very nicely with gnuplot
set terminal png size 640,480
set xlabel 'concurrency'
set ylabel 'millis'
set output 'tp99.png'
plot 'data.csv' using 2 with lines title 'tp99 response time'
set output 'mean.png'
plot 'data.csv' using 3 with lines title 'mean response time'
set output 'requests_per_second.png'
plot 'data.csv' using 4 with lines title 'requests/second'
Nothing super fancy, but is kind of fun :-)
Reworking the Atlas CLI 09 Mar 2012
The current CLI for Atlas was always a stopgap measure – something to let me invoke it on the command line during development. The time has come for a real user interface though. This post is really just me thinking through how this should work - it is rambly, but capturing the reasoning is helpful :-)
Project Centric
It has become apparent that Atlas is project-centric, such as git. A common and useful pattern for project centric stuff is to slap it all into a version-controllable directory, and provide a nice way to kickstart a new project. So, we have the beginning of our interactions:
$ atlas init
This will populate the current directory with a barebones atlas project, and initialize atlas’s state keeping. We’ll encourage the seperation of environment and system model, so let’s drop two files, a system model and a development environment description. As Atlas exists right now, that would be something like system.rb and env-dev.rb.
System and Environment Model Definitions
These two things, system model and environment description, are the heart of an atlas project so we want them to be front and center when you look at a project. I’d like to drop them right in the root of the project, but this conflicts with another desire – that we be able to detect and load system models and environment descriptors for a project.
Once a system starts growing, but before it really has a number of teams working on it or using it, I expect it will be very common to drop all the parts of the system into one project via seperate files, organized as makes sense for the system. Right now that is awkward, any files aside from the root must be explicitely included as external elements in the system model. This works but in the case of the environment descriptor, when you have multiple environments, means checking out the project multiple times and initializing each checkout for a different environment. Yuck.
To complicate it further, it is becoming apparent that in order to make systems composable from external models (ie, if Basho were to provide a descriptor for a Riak system, or Puppet Labs for a Puppet it would be nice to just have to reference one URL for it. It is convenient to allow one file to describe both the environment and the system in this case. Additionally, it has become apparent that sometimes an environment needs to define system elements. In the EC2 case, for example, this might be creating a VPN instance so the machine running atlas can connect to instances inside a security group.
Given that we want to have arbitrary environment and system models for a project, we’d like them to take center stage in the root of the project, and finally we migh want other convenience scripts in the root of the project, one option seems to be just changing their extension and globbing them together to load – ie, system.atlas and dev-env.atlas.
Given that atlas descriptors are ruby files, this is inconvenient for automatic file type detection for code highlighting, which is unfortunate.
Another option is to give up on the root and drop a model/ directory which contains the system and environment descriptors. Model is an overloaded term, due use of the word to describe database access code in Rails-inspired web frameworks. In this case, it is used to represent a model of the system and environments in which the system will run.
Using a model/ directory gives a slight affordance to new users in that when the open a descriptor, it is nicely highlighted. This is small, but frankly really handy. I don’t have my .emacs on every machine I may need to muck around with models from and syntax highlighting really helps with reading this stuff.
On descriptor location, looking way out, Atlas internally creates tree structures for its models and operates wholly on those trees. That the current implementation uses ruby to describe those models is basically a convenience. Personally, I think S-expressions model trees even more nicely, and could see using a lisp dialect as well, or a custom syntax. Taking ruby and calling it .atlas implies more authority than it deserves.
Given all this, I think I’ll go with the model/ directory approach. Given that, our newly initialized project now looks like
brianm@ufo:/tmp/ratless$ atlas init
brianm@ufo:/tmp/ratless$ find .
.
./model
./model/dev-env.rb
./model/system.rb
./README
brianm@ufo:/tmp/ratless$
Running Atlas For Real
The primary action of atlas, and what folks will most often do with it, is to converge a system instance on the model. Right now this is
brianm@ufo:/tmp/ratless$ atlas update
I am not sure the best command for this. I think of it as convergence, so am tempted to switch the command to converge. Probably I’ll just make them aliases for the same command, but I’ll need to sort out the canonical one for docs.
This brings up another change needed – a raw atlas
converge works today because a project is initialized with the environment model to use. If a given project is going to support running against multiple environments we’ll need a way to pick which one to run against. To do this reasonably we’ll need to make the default configurable (the default will default to an environment named dev I think), which means a configuration system.
Configuration
Today atlas uses SQLite for all of its state, but this is a crap option for configuration. Configuration should be human mungeable text. So, we need to add a human mungeable text file.
I like simple sectioned key/value pairs for configuration, personally, a la the format used by Python’s ConfigParser. The other format I really like is Lua. An interesting alternative is Typesafe’s config library. The typesafe library provides a very readable curly-based block/hierarchical syntax in addition to straight key/value pairs and json, but json isn’t really human-friendly writeable IMHO). Using simpler things, like straight key/value pairs or sectioned key/value pairs encourages configuration to stay very simple, which is appealing.
Atlas is designed to be extended with various provisioners, installers, and listeners specialized for different environments or tools. Frequently, extensions will need configuration as well, and some extensions may need (or just have because some developers work that way) extensive and complex configuration. While big complex configuration is pretty distasteful, it can happen for valid reasons. If it can happen, we should at least steer it towards something consistent, which is an argument in favor of something like the Typesafe library.
Putting aside configuration syntax (and semantics I guess, in that there is at least a choice between hierarchical and flat here) there is where to put it. The clear trend is to hide it from plain sight – stick it in a .$file of some sort. This works if there is no expectation of changing configuration very often (emacs), or if there is an expectation of using tooling to change it (git). Personally, I dislike using tooling to change configuration, so will steer towards just pointing folks to the file.
Atlas presently uses a .atlas/ directory to keep application managed state – the aforementioned SQLite database, generated ssh keys, etc. This directory is only semi-private – while there is no expectation of users poking into the SQLite database, use of generated SSH keys is expected and okay. One options is to drop an atlas.conf file here. Putting it here implies that we don’t want users to muck with it often, and that out of the box users should not need to muck with it at all.
Another alternative is to drop an atlas.conf in the root. This implies that users can and should make changes to configuration and it deserves to be highlighted.
The last option I want to consider is a conf/ directory analogous to the models directory. This makes it easier to support multiple configuration file formats, etc, which if we take the Typesafe config approach might be helpful.
Conceptually, configuration should be used to control Atlas’s behavior within the project. The project itself should be the main focus of users’ focus, and I don’t expect configuration to be changed that often. As long as we pick defaults well then most users shouldn’t need to touch the Atlas progam configuration. Given this, I like the .atlas/atlas.conf option the most.
Back to Running Atlas
Okay, so we have a sketch for configuration. We will default to running in an environment named dev (not from the file name, but from the environment declaration:
env "dev" do
# ...
end
env "prod" do
# ...
end
env "load" do
# ...
end
and we’ll process all the descriptors in models/ (location changeable via .atlas/atlas.conf) which can define multiple environments of which only one will be used. Using something other than the default can be either an argument or an option to converge:
$ atlas update
$ atlas update -e load
$ atlas update load
The first would operate on the default (which defaults to dev the other two would operate on the load environment. I am not sure which form I like, argument or option. Need to ponder.
Java Daemonization with posix_spawn(2) 24 Jan 2012
The traditional way of daemonizing a process involves forking the process and daemonizing it from the current running state. This doesn’t work so well in Java because the JVM relies on several additional worker threads, and fork only keeps the thread calling fork. So, basically, you need to start a new JVM from scratch to create a child process.
The traditional way of launching a new program is to fork and exec the program you wish to start. Sadly, this also fails on Java because the calls to fork and exec are seperate, non-atomic (in the platonic sense, not the JMM sense) operations. There is no guarantee that the exec will be reached, or that the memory state of the JVM will even be sound when the exec is reached, because you could be in the middle of a garbage collection and pointers could be all over. In practice, this would happen exceptionally rarely, at least. Charles Nutter has talked about this problem in JRuby as well.
The best method I know of to daemonize a Java process, then, is to use posix_spawn. This function launches a new process based on the program image passed to it – it is like fork and exec in one system call. Over the weekend I took some time to put together a small library to do this, Gressil using jnr-ffi. It turned out that the hardest part, by far, was reconstructing the full ARGV array.
On linux, with ProcFS it is pretty straightforward. On OS X it is supposed to be possible but all the code samples I have found seem to be based on that snippet, and they look forward into the “String area” which usually has ARGV laid out in the correct order, but sometimes doesn’t. Trying to look backward from the pointer returned by the sysctl would be straightforward in C (decrement the pointer) but jnr-ffi seems to copy the memory space into a Java byte array, so there is no going backward. Attempting to guess how far back to jump the pointer to get a new block of memory usually segfaults for me :-) Anyone with thoughts, a version based on the above reference is in Gressil, MacARGVFinder.
Anyway, it turns out you can get most of this information from the Sun JVM. The only dodgy part of that is the program arguments (the stuff passed into Java’s public static void main(String[] args) method. Luckily, those are passed to main, so we can just ask for them in the library.
Given that, you can use Gressil to daemonize Java processes via re-launching like so:
package org.skife.gressil.examples;
import org.skife.gressil.Daemon;
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import java.util.Date;
import static org.skife.gressil.Daemon.remoteDebugOnPort;
public class ChattyDaemon
{
public static void main(String[] args) throws IOException
{
new Daemon().withMainArgs(args)
.withPidFile(new File("/tmp/chatty.pid"))
.withStdout(new File("/tmp/chatty.out"))
.withExtraMainArgs("hello", "world,", "how are you?")
.withExtraJvmArgs(remoteDebugOnPort(5005))
.daemonize();
while (!Thread.currentThread().isInterrupted()) {
System.out.println(new Date() + " " + Arrays.toString(args));
try {
Thread.sleep(1000);
}
catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
}
In the parent process the call to Daemon#daemonize() will call System.exit(), in the child process it will return normally.
The child process, in this case, will also listen for a Java debugger to attach on port 5005. It will attach stdout to /tmp/chatty.out, and stdin and stderr will default to /dev/null (which stdout would also attach to by default if it were not specified).
I cut a 0.0.1 release which should be synced in maven central by now as org.skife.gressil:gressil:0.0.1 by now. It should work pretty much anywhere with a posix-compliant libc, but I have only extensively tested on OS X.