A few months ago, the awesome provider Init7 has released their awesome FTTH offering Fiber7 which provides synchronous 1GBit/s access for a very fair price. Actually, they are by far the cheapest provider for this kind of bandwith.

Only cablecom comes close at matching them bandwidth wise with their 250Mbits package, but that's 4 times less bandwith for nearly double the price. Init7 also is one of the only providers who officially states that their triple-play strategy is that they don't do it. Huge-ass kudos for that.

Also, their technical support is using Claws Mail on GNU/Linux - to give you some indication of the geek-heaven you get when signing up with them.

But what's really exciting about Init7 is their support for IPv6. In-fact, Init7 was one of the first (if not the first) providers to offer IPv6 for end users. Also, we're talking about a real, non-tunneled, no strings attached plain /48.

In case that doesn't ring a bell, a /48 will allow for 216 networks consisting of 264 hosts each. Yes. That's that many hosts.

In eager anticipation of getting this at home natively (of course I ordered Fiber7 the moment I could at my place), I decided to play with IPv6 as far as I could with my current provider, which apparently lives in the stone-age and still doesn't provide native v6 support.

After getting abysmal pings using 6to4 about a year ago, this time I decided to go with tunnelbroker which these days also provides a nice dyndns-alike API for updating the public tunnel endpoint.

Let me tell you: Setting this up is trivial.

Tunnelbroker provides you with all the information you need for your tunnel and with the prefix of the /64 you get from them and setting up for your own network is trivial using radvd.

The only thing that's different from your old v4 config: All your hosts will immediately be accessible from the public internet, so you might want to configure a firewall from the get-go - but see later for some thoughts in that matter.

But this isn't any different from the NAT solutions we have currently. Instead of configuring port forwarding, you just open ports on your router, but the process is more or less the same.

If you need direct connectivity however, you can now have it. No strings attached.

So far, I've used devices running iOS 7 and 8, Mac OS X 10.9 and 10.10, Windows XP, 7 and 8 and none of them had any trouble reaching the v6 internet. Also, I would argue that configuring radvd is easier than configuring DHCP. There's less thought involved for assigning addresses because autoconfiguration will just deal with that.

For me, I had to adjust how I'm thinking about my network for a bit and I'm posting here in order to explain what change you'll get with v6 and how some paradigms change. Once you've accepted these changes, using v6 is trivial and totally something you can get used to.

  • Multi-homing (multiple adresses per interface) was something you've rarely done in v4. Now in v6, you do that all the time. Your OSes go as far as to grab a new random one every few connections in order to provide a means of privacy.
  • The addresses are so long and hex-y - you probably will never remember them. But that's ok. In general, there are much fewer cases where you worry about the address.
    • Because of multi-homing every machine has a guaranteed static address (built from the MAC address of the interface) by default, so there's no need to statically assign addresses in many cases.
    • If you want to assign static addresses, just pick any in your /64. Unless you manually hand out the same address to two machines, autoconfiguration will make sure no two machines pick the same address. In order to remember them, feel free to use cute names - finally you got some letters and leetspeak to play with.
    • To assign a static address, just do it on the host in question. Again, autoconfig will make sure no other machine gets the same address.
  • And with Zeroconf (avahi / bonjour), you have fewer and fewer oportunities to deal with anything that's not a host-name anyways.
  • You will need a firewall because suddenly all your machines will be accessible for the whole internet. You might get away with just the local personal firewall, but you probably should have one on your gateway.
  • While that sounds like higher complexity, I would argue that the complexity is lower because if you were a responsible sysadmin, you were dealing with both NAT and a firewall whereas with v6, a firewall is all you need.
  • Tools like nat-pmp or upnp don't support v6 yet as far as I can see, so applications in the trusted network can't yet punch holes in the firewall (what is the equivalent thing to forwarding ports in the v4 days).

Overall, getting v6 running is really simple and once you adjust your mindset a bit, while stuff is unusual and taking some getting-used-to, I really don't see v6 as being more complicated. Quite to the contrary actually.

As I'm thinking about firewalls and opening ports, actually, as hosts get wiser about v6, you actually really might get away without a strict firewall as hosts could grab a new random v6 address for every connection they want to use and then they would just bind their servers to that address.

Services binding to all addresses would never bind to these temporary addresses.

That way none of the services brought up by default (you know - all those ports open on your machine when it runs) would be reachable from the outside. What would be reachable is the temporary addresses grabbed by specific services running on your machine.

Yes. An attacker could port-scan your /64 and try to find the non-temporary address, but keep in mind that finding that one address out of 264 addresses would mean that you have to port-scan 4 billion traditional v4 internets per attack target (good luck) or randomly guessing with an average chance of 1:263 (also good luck).

Even then a personal firewall could block all unsolicited packets from non-local prefixes to provide even more security.

As such, we really might get away without actually needing a firewall at the gateway to begin with which will actually go great lengths at providing the ubiquitous configuration-free p2p connectivity that would be ever-so-cool and which we have lost over the last few decades.

Me personally, I'm really happy to see how simple v6 actually is to get implemented and I'm really looking forward to my very own native /48 which I'm probably going to get somehwere in September/October-ish.

Until then, I'll gladly play with my tunneled /64 (for now still firewalled, but I'll investigate into how OS X and Windows deal with the temporary addresses they use which might allow me to actually turn the firewall off).

Read on →

Last autumn, I was talking about how I would like to see pdo_pgsql for PHP to be improved.

Over the last few days I had time to seriously start looking into making sure I get my wish. Even though my C is very rusty and I have next to no experience in dealing with the PHP/Zend API, I made quite a bit of progress over the last few days.

First, JSON support

screenshot showing off json support

If you have the json extension enabled in your PHP install (it's enabled by default), then any column of data type json will be automatically parsed and returned to you as an array.

No need to constantly repeat yourself with json_parse(). This works, of course, with directly selected json columns or with any expression that returns json (like array_to_json or the direct typecast shown in the screenshot).

This is off by default and can be enabled on a per-connection or a per- statement level as to not break backwards compatibility (I'll need it off until I get a chance to clean up PopScan for example).

Next, array support:

screenshot showing off array support

Just like with JSON, this will automatically turn any array expression (of the built-in array types) into an array to use from PHP.

As I'm writing this blog entry here, this only works for text[] and it's always enabled.

Once I have an elegant way to deal with the various types of arrays and convert them into the correct PHP types, I'll work on making this turnoffable (technical term) too.

I'll probably combine this and the automatic JSON parsing into just one setting which will include various extended data types both Postgres and PHP know about.

Once I've done that, I'll look into more points on my wishlist (better error reporting with 9.3 and later and a way to quote identifiers comes to mind) and then I'll probably try to write a proper RFC and propose this for inclusion into PHP itself (though don't get your hopes up - they are a conservative bunch).

If you want to follow along with my work, have a look at my pdo_pgsql-improvements branch on github (tracks to PHP-5.5)

Read on →

In the summer of 2012, I had the great oportunity to clean up our hosting infrastructure. Instead of running many differently configured VMs, mostly one per customer, we started building a real redundant infrastructure with two really beefy physical database machines (yay) and quite many (22) virtual machines for caching, web app servers, file servers and so on.

All components are fully redundant, every box can fail without anybody really needing to do anything (one exception is the database - that's also redundant, but we fail over manually due to the huge cost in time to failback).

Of course you don't manage ~20 machines manually any more: Aside of the fact that it would be really painful to do for those that have to be configured in an identical way (the app servers come to mind), you also want to be able to quickly bring a new box online which means you don't have time to manually go through the hassle of configuring it.

So, In the summer of 2012, when we started working on this, we decided to go with puppet. We also considered Chef but their server was really complicated to set up and install and there was zero incentive for them to improve because that would, after all, disincentivse people from becoming customers of their hosted solutions (the joys of open-core).

Puppet is also commerically backed, but everything they do is available as open source and their approach for the central server is much more «batteries included» than what Chef has provided.

And finally, after playing around a bit with both Chef and puppet, we noticed that puppet was way more bitchy and less tolerant of quick hacks around issues which felt like a good thing for people dabbling with HA configuration of a multi machine cluster for the first time.

Fast forward one year: Last autumn I found out about ansible (linking to their github page - their website reads like a competition in buzzword-bingo) and after reading their documentation, I immediately was convinced:

  • No need to install an agent on managed machines
  • Trivial to bootstrap machines (due to above point)
  • Contributors don't need to sign a CLA (thank you so much, ansibleworks!)
  • No need to manually define dependencies of tasks: Tasks are run requentially
  • Built-in support for cowsay by default
  • Many often-used modules included by default, no hunting for, say, a sysctl module on github
  • Very nice support for rolling updates
  • Also providing a means to quickly do one-off tasks
  • Very easy to make configuration entries based on the host inventory (which requires puppetdb and an external database in the case of puppet)

Because ansible connects to each machine individually via SSH, running it against a full cluster of machines is going to take a bit longer than with puppet, but our cluster is small, so that wasn't that much of a deterrent.

So last Sunday evening I started working on porting our configuration over from puppet to Ansible and after getting used to the YAML syntax of the playbooks, I made very quick progress.


Again, I'd like to point out the excellent, built-in, on-by-default support for cowsay as one of the killer-features that made me seriously consider starting the porting effort.

Unfortunately though, after a very promising start, I had to come to the conclusion that we will be sticking with puppet for the time being because there's one single feature that Ansible doesn't have and that I really, really want a configuration management system to have:

I'ts not possible in Ansible to tell it to keep a directory clean of files not managed by Ansible in some way

There are, of course, workarounds, but they come at a price too high for me to be willing to pay.

  • You could first clean a directory completely using a shell command, but this will lead to ansible detecting a change to that folder every time it runs which will cause server restarts, even when they are not needed.

  • You could do something like this stack overflow question but this has the disadvantage that it forces you into a configuration file specific playbook design instead of a role specific one.

What I mean is that using the second workaround, you can only have one playbook touching that folder. But imagine for example a case where you want to work with /etc/sysctl.d: A generic role would put some stuff there, but then your firewall role might put more stuff there (to enable ip forwarding) and your database role might want to add other stuff (like tweaking shmmax and shmall, though that's thankfully not needed any more in current Postgres releases).

So suddenly your /etc/sysctl.d role needs to know about firewalls and databases which totally violates the really nice separation of concerns between roles. Instead of having a firewall and a database role both doing something to /etc/sysctl.d, you know need a sysctl-role which does different things depending on what other roles a machine has.

Or, of course, you just don't care that stray files never get removed, but honestly: Do you really want to live with the fact that your /etc/sysctl.d, or worse, /etc/sudoers.d can contain files not managed by ansible and likely not intended to be there? Both sysctl.d and sudoers.d are more than capable of doing immense damage to your boxes and this sneakily behind the watching eye of your configuration management system?

For me that's inacceptable.

So despite all the nice advantages (like cowsay), this one feature is something that I really need and can't have right now and which, thus, forces me to stay away from Ansible for now.

It's a shame.

Some people tell me that implementing my feature would require puppet's feature of building a full state of a machine before doing anything (which is error- prone and frustrating for users at times), but that's not really true.

If ansible modules could talk to each other - maybe loosly coupled by firing some events as they do stuff, you could just name the task that makes sure the directory exists first and then have that task register some kind of event handler to be notified as other tasks touch the directory.

Then, at the end, remove everything you didn't get an event for.

Yes. This would probably (I don't know how Ansible is implemented internally) mess with the decouplling of modules a bit, but it would be so far removed from re-implementing puppet.

Which is why I'm posting this here - maybe, just maybe, somebody reads my plight and can bring up a discussion and maybe even a solution for this. Trust me: I'd so much rather use Ansible than puppet, it's crazy, but I also want to make sure that no stray file in /etc/sysctl.d will bring down a machine.

Yeah. This is probably the most words I've ever used for a feature request, but this one is really, really important for me which is why I'm so passionate about this. Ansible got so f'ing much right. It's such a shame to still be left unable to really use it.

Is this a case of xkcd1172? Maybe, but to me, my request seems reasonable. It's not? Enlighten me! It is? Great! Let's work on fixing this.

Read on →

Like many others, I couldn't wait for Apple to finally upgrade their MacPro and like many others, when they could finally be ordered, I queued up to get mine.

Last Monday, after two months of wait, the package finally arrived and I could start playing with it. I have to say: The thing is very impressive.

The hardware itself is very lightweight and compact. Compared to the old aluminium MacPro it was replacing, it felt even smaller than it is. Also, the box is nearly silent - so silent in fact, that now the hum of the dimmed background light in my old 30" Cinema Display is louder than the machine itself.

Speaking of that 30" display: It's using a dual-link DVI port. That means a special adapter is required to connect it to the new Thunderbolt ports - at least if you want to use a higher resolution than 1280x800 (which you definitely do).

The adapter is kinda difficult to get, especially as I totally forgot about it and I reall wanted to migrate to the new machine, so I had to look through local retail (only the one from Apple even remotely available) as opposed to Amazon (three other models available, some cheaper).

The device is huge by the way. I'm sure there's some electronics in there (especially when you consider that you have to plug it into a USB port for power), probably to split the full 2560x1600 pixels sent over Thunderbolt into two images of 1280x800, only to be reassembled in the display I guess.

The fact that there obviously is processing going on leaves a bit of a bad taste as it's one more component that could now break and, of course, there might be display lag or quality degradation.

At some time, there was for sure, if the adapters reviews are to be believed, but so far, I wasn't able to notice bad quality nor lag, but the fact that now there's one more active component involved in bringing me a picture makes me just a tad bit nervous.

Anyways - let's talk about some more pleasant things.

One is the WiFi: With the old MacPro I had peak transfer of about 3 MBytes/s which was just barely good enough for me to not wanting to go through the trouble of laying cable, even though it really pissed me off at times.

On the new Pro, I reached 18 MBytes/s over the exact same WiFi yesterday which removes any need for ever considering installing a physial cable. Very handy. Remember: It's not a file server, it doesn't run a torrent client, it doesn't serve movies to my home network. The really large bulk transfers it does are mainly caused by Steam which clearly is the bottleneck here (it never manages to saturate my 150MBit/s downstream).

Another thing that really surprises me is the sleeping behavior of the box. Well, actually, the waking up behavior: When asleep, the thing wakes up instantly (less than a second) - never in my live have I seen such a quick waking up from sleep in a computer.

Yes. I'm waiting for the fan to spin down and all audible noise to go away, but still. Hit any key on the keyboard and the machine's back. We're talking "waking an iphone from sleep" speeds here.

It might be that the machine has multiple levels of sleep states, but the instant wake-up also happens after sleeping for over 12 hours at which point a deeper sleep would totally make sense if there was any.

What is strange though: I seem to be able to wake the machine by pinging it. Yes. I know about the bonjour proxy, but in this case, I'm pinging it directly by IP and it wakes up (the first ping has a roundtrip time for 500ish ms - yes. it wakes THAT quickly).

This leads me to believe that the machine might not actually be sleeping for real though because waking from a direct ping requires quite a bit more technology than waking from a WOL packet.

Somdeday, I'll play with tcpdump to learn what's going on here.

Performance-wise, I haven't done that much testing, but replaying a test Postgres database dump that takes 5ish minutes on a 2012 retina MacBook Pro completes in 1:12 minutes on the pro - pretty impressive.

And one last thing: When you get a machine as powerful as this, there's of course also the wish of playing a game or two on it. As I had one SSD dedicated to Bootcamp in the old Pro, I was curious whether I might be able to keep this setup: The built-in flash drive dedicated to MacOS and Windows on its own (the old one) dedicated SSD.

Now that we don't have internal drive bays any more, this might seem tricky, but yesterday, I managed to install Windows 8 nicely on that SSD after connecting it via Thunderbolt using this adapter (no affiliate code - I got the link straight from google).

I guess the fact that it's using Thunderbolt makes Windows think it's a built-in hard drive which is what makes this work: You're not allowed to install Windows on a portable drive due to licensing issues.

The adapter is not actually intended for use with arbitrary drives (it's an accessory to some Seagate portable drives), but it works totally well and is (physically) stable enough. I'll have to do a bit of benchmarking to see how much performance I lose compared to the old built-in solution, but it certainly doesn't feel any slower.

Overall, I'm really happy with my new toy. Yes, it's probably overpowered for my needs, but it's also cool has hell, it is the first MacPro I own where sleep works reliably (though I'm inclined to say that it works suspiciously well - it might be cheating) and the fact that bootcamp still works with a dedicated external drive makes me really happy too.

Read on →