So I’m tempted. It looks really slick.

There’s just a few things bothering me. Please bear in mind that I have done no active research. I go by twitter chit chat, billboards and window displays. I don’t watch cable tv or look at banner ads.

I’m worried about essential apps. Will gmail work? I’m not switching to hotmail. How about twitter, facebook, über, zipcar, sonos? I don’t hear people raving about windows phone clients.

I mean, I don’t need everything. Just those essentials..

Contract. The phone is cheaper, granted, but I think they’re still pushing me into a contract. But what if it’s a turkey? I don’t want to be stuck with the same turkey for two Christmases.

You know how cool it is to watch movies on your four inch phone? Honestly? It’s rubbish. Rubbish. So how about you throw me a smaller, sexier phone that fits in my pocket? Get freaky on me. I think your styles can handle it. I won’t watch more than YouTube videos of cats anyway.

Finally, my android phone. Everything about it is annoying me now. I’ve got a Nexus One, and I’m so over it. Every time I dock it, I curse the stupid delay, the lack of notifiers on the screensaver, the fact that I can’t differentiate ringtones between my wife’s urgent call and the bastard automated calls. That damnable lock screen. I hate this phone.

Ok, this is irrational, because the phone is fine, and I’m just tired of the quirks. Understood.

But to every one of those points, I look to the iPhone and sigh. How much better will life be when I have an iPhone? Maybe I just can’t possibly bear to buy anything else.

So to Microsoft/Nokia I say: reassure me. Give me billboards, give me window displays. I’m ready to take the jump.

Promise me it’ll be ok.

This is true. You do figure it out.

But, hey, it might be that you’re visiting a friend with a baby (like us for example, please come and visit), and you haven’t had your own sprog yet. How do you behave? Here is a handy guide.

Say hello to mum. If not first, then at least second. It’s easy to forget, what with the baby.

Wash your hands before touching the baby. Golden rule, no exceptions.

Don’t take the baby. Ask, if you want a hold. But not straight away. Let the baby get used to you. Babies have a phase called “stranger anxiety”. If the baby screams at you, back off and give it time.

If the baby cries, keep clear. This can be a stressful time for the parents, especially with a guest present. Don’t hover, don’t help, just stand back and amuse yourself. Maybe take a walk.

Don’t get all up in the baby’s face. You don’t shove your face into every strangers nose. He can see you. There’s really no need to be less than an inch away.

If you find yourself in the kitchen, tidy up and wash up. Observe the local practises (there may be equipment just for baby stuff). Try to find yourself in the kitchen regularly.

Finally, here’s the best point of all. It’s the best. The winner.
“How can I help?”
Simple question. Ask it often.

We parents will thank you for following these rules. If not out loud (we’re busy), then in spirit.

We thank you.

• XP2012 is in Neo Technology’s hometown of Malmo, Sweden [link]
• The BCS CMSG conference program is also out [link]
• VMWare sent a reminder for VMWare Forum 2012 next week  [link]

News:

• Rackspace are blinging up their cloud product, and adding cloud databases [link]
• Cloudbees are doing a webinaar on ALM for PAAS [link]

News, 5 May is a post from: The Build Doctor. Sponsored by AnthillPro, the build and deployment automation server that lets you release with confidence.

I don’t really miss the long run to work though. It’s a slow four miles along the coast, avoiding traffic lights and the famous San Franciscan hills. I just jog at a steady pace. The distance isn’t hard, just the boredom. Long distance running is pretty dull – you have to have a lot on your mind to be able to run without getting really bored.

It’s running fast that’s more fun.
Not a run, not full speed even.
I mean a full-on sprint.

That extra burst of energy you throw in when you’re already going full speed. When your arms are pumping as hard as your legs, reaching out to claw the air closer. When the skin on your face starts moving against the bone. When you shift your weight down and back, so your feet can make maximum contact with the ground, but it’s still not enough. When you forget about breathing because it’s just another distraction. When every muscle in your body is working maximum power, maximum speed, just to propel you forward.

That’s the kind of running I miss.

In turn, the Sun will orbit the centre of the Milky Way (our galaxy).
This will take 230,000,000 years.
About three million lifetimes. Nine million generations.
In the entire history of modern man, we’ve barely started an orbit.

Our galaxy doesn’t orbit anything. But then, the Universe is only 13,700,000,000 years old. Maybe we just haven’t had a chance to get started.

The atoms in our bodies are elements that can only be created in supernovae – exploding stars. Going by the quantities we have available, we’ve probably been through two supernovae.

Hot gas gathers into a ball of fire, nuclear explosion, drifts apart, gathers into clouds, collects into ball of fire, nuclear explosion, drifts apart, gathers into clouds, collects into ball of fire. Our solar system.

Our sun is about 5,000,000,000 years old. The previous stars from which we’re made were bigger, and the universe was smaller and hotter, so they didn’t last as long. We can last another 5,000,000,000 years before we have to start looking for a new one. Our sun won’t explode – it will just fizzle out.

The heavens move, but our lifetimes are just a beat of the hummingbird’s wing.

Our nearest neighbour is 4 light years away. Not so far, at the speed of light. Of course, we can’t travel at the speed of light. We can mathematically prove that we’ll never be able to come close. One light year is 6,000,000,000,000 miles.

The stars are unreachable.

Despite all our advances in science and technology over the last century, it still takes an immense amount of power to lift anything into orbit. And mere words can’t convey the scale of the power required. These are the biggest machines mankind has ever created. They burn simple chemical fuels; no atomic power is used.

The problem of getting into orbit is still an engineering challenge, not a scientific challenge.

From orbit, where can we go? The moon is nearby, for want of a better word. It’s a rock.

Venus and mercury burn so hot, they’re unapproachable. Mars is an arid desert. We can no more populate that than we can the heart of the Sahara.

The gas giants are so distant that they’d take years to reach, and are incredibly dangerous. Jupiter’s magnetic field is so huge that it’s tearing the moons apart. The winds of Saturn orbit once every ten hours – that’s thousands of miles an hour,

There is nowhere for us to go.

Space is a beautiful, fascinating light show, forever beyond our reach.
Tragically boring.

–

If you’re interested in space, you should sponsor my friend Chris’s project for Astronomers Without Borders: reach for the stars.

I think it’s a common question at Apple too.
Their UI design is full of it.

Sometimes for the good:
“how do I get rid of this virus?”
“virus? why do you have a virus?”
“I downloaded this software…”
“ah, let me build you an app store”

Sometimes for the bad:
“why doesn’t this green button work?”
“what is the problem you’re trying to solve?”
“I want to maximise the window”
“ah, let me build you Spaces, a whole new way to…”
“just make the fucking button work”

Generally I am very glad I started this process, I am actually continuing to use it today which is probably the longest I’ve ever used any TODO or reminder application. It really is good to be able to hack your own workflow and have a tool you can adapt to your needs rather than try to fit the mold of some off the shelf tool.

The time tracking feature has proven very valuable – as I work on my various projects I will track time worked with gwtf and review my mental estimate at the end of adding a feature. In time I am sure this will improve my ability to provide accurate time estimates while coding something for work etc.

The big thing I’ve added is a reminder system that can send notifications via email, boxcar or notifo (community contributed method). The image show my iPhone receiving a push message. All items can have a reminder date and in line with the Unix CLI approach this is done using your system at(1) command. Each notification can go to multiple recipients so I get email and push notifications.

% gwtf new this is a test --remind="now + 1 week" --done --ifopen
Creating reminder at job for item 30: job 46 at 2012-03-13 20:09
   30                 this is a test

Here I will get a reminder a week from now after reminding the item will be marked as done. The reminder will only be sent if item has not already been closed.

Building on this I added a special project called reminders that does not show up in the normal list output, this project is where simple one-off reminders (and soon repeating ones) go. Being hidden from the list output means I can have many of them without feeling like I have a huge TODO backlog since these aren’t strictly TODO items.

% gwtf remind --at="now +1 hour" do something
Creating reminder at job for item 84: job 66 at 2012-04-10 15:11
   103 L   2012-04-13 do something

Items now have due dates and there is a notification method that will email you all due and overdue tasks. I find if my todo list get mailed to me every day I get blind to it real quick, by only mailing due and overdue items they stand out and I pay them attention.

Various item list commands have been added. When I log into my shell I get the following, the colors show me that the projects have items due soon, it would go red when they are overdue.

I’ve added an overview mode to the list command that shows all projects and their open items, again color coded as above.

I’ve also augmented all the date processing using the excellent Chronic gem. This means anywhere I need a date or time specification I can use natural language dates. For example the due date specification can be as simple as –due=”next week” which would end up being due on next Wednesday. I can be more specific like –due=”7pm next tuesday” etc. I really like this mode of date input since I almost never know what the date is anyway it’s a big challenge to type in full dates for this kind of system.

That’s the big ticket items but there has been a ton of small tweaks. Overall I’ve done 16 releases of the Gem and it’s been downloaded 1600+ times from rubygems.org. I put a little website up for it using the new GitHub site system with full documentation etc.

I am not sure who the 1600 downloaders are, I am certainly not developing this with other peoples needs in mind but hopefully someone is gaining value from it.

Till now I’ve just done puppet runs via MCollective, basically I’d edit some puppet files and after comitting them just send off a puppet run with mcollective, supplying filters by hand so I only trigger runs on the appropriate nodes.

I started looking into git commit hooks to see if I can streamline this. I could of course just trigger a run on all nodes after a commit, there is no problem with capacity of masters etc to worry about. This is not very elegant so I thought I’d write something to parse my git push and trigger runs on just the right machines.

I’ll show a simplified version of the code here, the full version of the post-receive hook can be found here. I’ve removed the parse_hiera, parse_node and parse_modules functions from this but you can find them in the code linked to. To use this code you will need MCollective 2.0.0 that is due in a few days.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90

#!/usr/bin/env ruby
 
require 'rubygems'
require 'grit'
require 'mcollective'
 
include MCollective::RPC
 
@matched_modules = []
@matched_nodes = []
@matched_facts = []
 
# read each git ref in the push and process them
while msg = gets
  old_sha, new_sha, ref = msg.split(' ', 3)
 
  repo = Grit::Repo.new(File.join(File.dirname(__FILE__), '..'))
 
  commit = repo.commit(new_sha)
 
  case ref
    when %r{^refs/heads/(.*)$}
      branch = $~[1]
      if branch == "master"
        puts "Commit on #{branch}"
        commit.diffs.each do |diff|
          puts "    %s" % diff.b_path
 
          # parse the paths and save them to the @matched_* arrays
          # these functions are in the full code paste linked to above
          case diff.b_path
            when /^hieradb/
              parse_hiera(diff.b_path)
            when /^nodes/
              parse_node(diff.b_path)
            when /^common\/modules/
              parse_modules(diff.b_path)
            else
              puts "ERROR: Do not know how to parse #{diff.b_path}"
          end
        end
      else
        puts "Commit on non master branch #{branch} ignoring"
      end
  end
end
 
unless @matched_modules.empty? && @matched_nodes.empty? && @matched_facts.empty?
  puppet = rpcclient("puppetd")
 
  nodes = []
  compound_filter = []
 
  nodes << @matched_nodes
 
  # if classes or facts are found then do a discover
  unless @matched_modules.empty? && @matched_facts.empty?
    compound_filter << @matched_modules << @matched_facts
 
    puppet.comound_filter compound_filter.flatten.uniq.join(" or ")
 
    nodes << puppet.discover
  end
 
  if nodes.flatten.uniq.empty?
    puts "No nodes discovered via mcollective or in commits"
    exit
  end
 
  # use new mc 2.0.0 pluggable discovery to supply node list
  # thats a combination of data discovered on the network and file named
  puppet.discover :nodes => nodes.flatten.uniq
 
  puts
  puts "Files matched classes: %s" % @matched_modules.join(", ") unless @matched_modules.empty?
  puts "Files matched nodes: %s" % @matched_nodes.join(", ") unless @matched_nodes.empty?
  puts "Files matched facts: %s" % @matched_facts.join(", ") unless @matched_facts.empty?
  puts
  puts "Triggering puppet runs on the following nodes:"
  puts
  puppet.discover.in_groups_of(3) do |nodes|
    puts "   %-20s %-20s %-20s" % nodes
  end
 
  puppet.runonce
 
  printrpcstats
else
  puts "ERROR: Could not determine a list of nodes to run"
end

The code between lines 14 and 46 just reads each line of the git post-receive hook STDIN and process them, you can read more about these hooks @ git-scm.com.

For each b path in the commit I parse its path based on puppet module conventions, node names, my hiera structure and some specific aspects of my file layouts. These end up in the @matched_modules, @matched_nodes and @matched_facts arrays.

MCollective 2.0.0 will let you supply node names not just from network based discovery but from any source really. Here I get node names from things like my node files, file names in iptables rules and such. Version 2.0.0 also supports a new query language for discovery which we use here. The goal is to do a network discovery only when I have non specific data like class names – if I found just a list of node names I do not need to do go out to the network to do discovery thanks to the new abilities of MCollective 2.0.0

In lines 48 to 90 I create a MCollective client to the puppetd agent, discover matching nodes and do the puppet runs.

If I found any code in the git push that matched either classes or facts I need to do a full MCollective discover based on those to get a node list. This is done using the new compound filtering language, the filter will look something like:

/some_class/ or some::other::class or fact=value

But this expensive network wide discovery is only run when there are facts or classes matched out of the commit.

Line 72 will supply the combined MCollective discovered nodes and node names discovered out of the code paths as discovery data which later in line 85 will get used to trigger the runs.

The end result of this can be seen here, the commit matched only 5 out of my 25 machines and only those will be run:

$ git push origin master
Counting objects: 13, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (7/7), 577 bytes, done.
Total 7 (delta 4), reused 0 (delta 0)
remote: Commit on master
remote:     common/modules/mcollective/manifests/client.pp
remote:
remote: Files matched classes: mcollective::client
remote:
remote: Triggering puppet runs on the following nodes:
remote:
remote:    node1                node2            node3
remote:    node4                node5
remote:
remote: 5 / 5
remote:
remote: Finished processing 5 / 5 hosts in 522.15 ms
To git@git:puppet.git
   7590a60..10ee4da  master -> master

—

Hello.

This is my icebreaker, so I get the chance to introduce myself.

My name is Kenneth Kufluk. I am one hundred and five years old, and Dick Costolo hasn’t spoken to me in six months.

But let me start with the name…

My surname is Kufluk, something I’ve always had trouble with. It’s unusual, even in Poland, where it came from. At school I’d get endless taunts of “toughluck kufluk”, and all sorts of mispronounciations.
One person says “Kufluk”, the next says “Kuflux”, the next says “Kooflux”.
For five years of high school, I had half the school laughing at me for being called “Cuthbert”.
But it’s Kufluk. It’s always been Kufluk.

Or so I thought.
Last year I published by Grandparent’s memoirs, and right there, buried in his stories of soviet invasion, cattle trucks and labour camps, and all the horrors of those times, he says that it was originally pronounced “Kooflook”. So now we know. I still prefer Kufluk, but I’m more relaxed about it now.

There’s nothing relaxed about my first name. It’s Kenneth. Nobody calls me Ken. Or at least, not twice.

I’ve been Kenneth my whole life, from when I was born until my 105th birthday this year.

Now I know what you’re thinking. 105? Is that his real hair? What’s his secret?
Well, yes, the hair is real. But there’s a story.

About six years ago I was a different man. Young, full of energy.
I’d been round the world twice.
I’d played in unicycle hockey world championships in japan, china and switzerland.
I’d sprint any distance.
I’d be out every night at the bar, picking up … well, picking up pints of beer, if I’m honest.
I’d actually been described as “a coiled spring”.

And then one summer, I found myself waking up at night with pain in my legs. I’d just walk it off and go back to bed. I assumed it was too much running.
But then it got worse.
Before long, a three block walk to the shops felt like miles. I’d come home and lie face-down on the bed, not asleep, but just exhausted.
No more running.
I’d find myself stopping on the way home from work, unable to go any further, but unable to think of any other way to get home.
This was chronic fatigue, something my “pull yourself together” upbringing had never really believed in.

Life got really hard.
I cancelled my first marathon, gave up on buying a house, and postponed my part-time degree.
But I still went to work.

And that was difficult too – not for exhaustion, but the constant questions “what did you do last night”, “how was your weekend”. I was face-down in bed at home, thanks for asking.

Little by little, month by month, I got better.
I learned to push through it.
Sometimes it got worse again, but overall I improved.

By the following year, I could get around.
I found that running was somehow easier than walking, and much easier than standing around.
So I entered the marathon again.

It was 26 miles of hard work. But I used what I’d learned – that you can push through exhaustion.
And I finished.
And I bought that house, finished that degree, and even proposed to my girlfriend.
But somewhere I’d still lost about a third of my life.

Five years on, and I’m cured. I still don’t like standing up for long stretches, or walking a long way. It’s like someone has turned up the gravity.
Some days are better than others. But I’m ok.
I’d say it’s made me feel about five years older.

Which, ok, only makes me about 40.
The real reason I feel 105 is because I have a six month old baby, which means I haven’t slept properly for, oh, about a year.

And why hasn’t Dick spoken to me?
Well, maybe it’s because I named my baby [REACTED].

Or maybe it’s because we’ve never actually really met.
One of those. I’ll never be sure.

I’ve been running Puppet in a atypical way for some time now.

• No Puppet Master
• No distribution packaging
• No commit until I know something works
• Only test from the outside

The only thing I feel I need to expand on is the last: testing.  Obviously if you write Ruby code, you should rspec the hell out of it.  But should you test Puppet code?  It’s mostly a declarative language.  If you’re properly declaring the outcomes that you want, then it can be easy.  If too much logic creeps in, you’re doing it wrong or you should write a function or type – and you should rspec the hell out of that.  This approach has served me for years with decalarative build tools*.

I have no desire to go verify that Puppet does what I tell it.  But I do care about the outcome.  Also, I need to know that it runs on the target platform, as I use a MacBook.

So I wrote Skewer.   Skewer’s only job is to:

• Provision cloud machines (or connect to existing ones)
• Bootstrap Puppet (via shell scripts and rubygems)
• Run Puppet
• Optionally run Cucumber features at the end

That scratches my itch.  Skewer probably won’t scratch your itch if you run lots of nodes.  It works on Ubuntu, though adding support for other operating systems wouldn’t be too hard.  You may also like the Puppet Cloud Provisioner.

Skewer evolved from a Rakefile that I used to test my puppet code.  I set out to rewrite it over the Christmas period, and got the last feature passing on Friday.  Like my other open source project, I learned a lot while doing it.    Skewer has some wrinkles, but I use it in my day job, and I’ve managed to keep that so far.

https://github.com/builddoctor/skewer

http://rubygems.org/gems/skewer

* Okay, I actually do a little bit more.  I use Rake to run puppet parser validate on every .pp file in my project, and I use puppet-lint to catch howlers.

Skewer – a tool for provisioning cloud nodes with Puppet is a post from: The Build Doctor. Sponsored by AnthillPro, the build and deployment automation server that lets you release with confidence.