Product Market Fit Cycle

Found some interesting talks on startup growth and scaling.
So you’ve been thinking about creating a minimum viable product (MVP); have you thought about the minimum viable segment (MVS) of the market you are going to sell to?

Starting today, my approach changes. If I could start tonight what would I create that would get the customer to do something right away?

Find the full gist on Michael J Skok here.

Running a Daemon (self restarting) Process in Ubuntu – the easy way

I recently had to think about changing my cron jobs that run every 2 minutes to something more reliable.

Basically, running a cron job every other minute has the disadvantage of restarting the script from the beginning even when the previous instance is still running. Say for example you have a script that needs to check if there are new records in a database table. You run that script every minute to “poll” the database, edit the records and move them to a new table.

What happens if you run the script at 2:00am and it doesn’t stop, then your cron runs again at 2:02am while the first one is still running? You will get two copies of the data in the database table.

This could easily corrupt your data and lead to duplicate results as seen above.Now this is where daemons and queue managers come in handy.

In this scenario I decided it was time for me to take the gauntlet of doing something different with queued jobs. While I’m still implementing this I thought it would be useful to keep a reference to the article that helped me make that decision. Find a piece of it below. Source link at the bottom.

 

Writing an upstart script

Turns out, writing your own upstart scripts is way easier than building init.d files based on the /etc/skeleton file.

Ok so here’s how it looks like; You should store the script in /etc/init/yourprogram.conf, create one for each Node program you write.

description "node.js server"
author      "kvz - http://kevin.vanzonneveld.net"

# used to be: start on startup
# until we found some mounts weren't ready yet while booting:
start on started mountall
stop on shutdown

# Automatically Respawn:
respawn
respawn limit 99 5

script
    # Not sure why $HOME is needed, but we found that it is:
    export HOME="/root"

    exec /usr/local/bin/node /where/yourprogram.js >> /var/log/node.log 2>&1
end script

post-start script
   # Optionally put a script here that will notifiy you node has (re)started
   # /root/bin/hoptoad.sh "node.js has started!"
end script

Wow how easy was that? Told you, upstart scripts are childsplay. In fact they’re so compact, you may find yourself changing almost every line cause they contain specifics to our environment.

non-root

Node can do a lot of stuff. Or break it if you’re not careful. So you may want to run it as a user with limited privileges. We decided to go conventional and chose www-data.

We found the easiest way was to prepend the Node executable with a sudo like this:

exec sudo -u www-data /usr/local/bin/node

Don’t forget to change your export HOME accordingly.

Restarting your Node.js daemon

This is so ridiculously easy..

$ start yourprogram
$ stop yourprogram

And yes, Node will already:

  • automatically start at boottime
  • log to /var/log/node.log

..that’s been defined inside our upstart script.

initctl

But wait, start and stop are just shortcuts. Who’s really behind the wheel here, is initctl. You can play around with the command to see what other possibilities there are:

$ initctl help
$ initctl status yourprogram
$ initctl reload yourprogram
$ initctl start yourprogram # yes, this is the same start
# etc

Update from October 30th, 2012

The basic idea has not changed since 2009, but we did add some tricks to our upstart script. Here’s what we now use in production at transloadit.com:

# cat /etc/init/transloaditapi2.conf
# http://upstart.ubuntu.com/wiki/Stanzas

description "Transloadit.com node.js API 2"
author      "kvz"

stop on shutdown
respawn
respawn limit 20 5

# Max open files are @ 1024 by default. Bit few.
limit nofile 32768 32768

script
  set -e
  mkfifo /tmp/api2-log-fifo
  ( logger -t api2 </tmp/api2-log-fifo & )
  exec >/tmp/api2-log-fifo
  rm /tmp/api2-log-fifo
  exec sudo -u www-data MASTERKEY=`cat /transloadit/keys/masterkey` /transloadit/bin/server 2>&1
end script

post-start script
   /transloadit/bin/notify.sh 'API2 Just started'
end script

Starting & stopping your daemon:

Usually, the suggested method for starting and stopping daemon process is to use the system’s “service” command. So here we would use:

sudo service transloaditapi2 start
sudo service transloaditapi2 stop

Check if your daemon is running:

sudo service transloaditapi2 status

via kvz.io.