HHVM 3.9.0 on Centos 7.1

So, I started a blog.

This is hosted on HHVM. The setup turned into a minor adventure. Business as usual on the bleeding edge.

Starting notes

  • HHVM does not have an official yum repo
  • The .rpm is still quite work in progress
    • They are working on it, or at least their TODO is extensive
      • What outright failed out of the box for me:
        • It did not create /var/lib/hhvm/
        • It did not create /var/log/hhvm/
        • It did not create /etc/tmpfiles.d/hhvm.conf
        • It did not create the group hhvm
        • It did not create the user hhvm
      • What could be better out of the box for me:
        • It only listens on TCP per default

Fixing from the bottom up

I simply created the user and the group in the way the package is supposed to. Notice here how it’s using /var/lib/hhvm as the home directory for the user. This was a very good hint later on.

groupadd -r hhvm
useradd -r -g hhvm -d /var/lib/hhvm -s /sbin/nologin -c "HHVM" hhvm

Unix sockets do have a reputation of being faster for within-system IPC, so tackling that came next.

pid = /run/hhvm/hhvm.pid

hhvm.server.file_socket = /run/hhvm/hhvm.sock
hhvm.server.type = fastcgi
hhvm.server.default_document = index.php
hhvm.source_root = /var/www

hhvm.log.use_log_file = true
hhvm.log.file = /var/log/hhvm/error.log

At this point, the daemon did not start as it cannot write to its socket. A bit of further housekeeping was required:

  • The log directory needs to exist and be writable
    • This one is not a fatal problem, but should be addressed if logging is desired
  • The run directory needs to exist and be writable
  • The run directory needs to persist across reboots
    • Option 1 – modify the systemd hhvm.service file
      • You have to upkeep it vs. future upstream changes in hhvm.service
      • ExecStartPre hackery is usually considered ugly these days
    • Option 2 – use /etc/tmpfiles.d/
      • This is a freedesktop.org standard for solving the problem scope
      • Check man tmpfiles.d if in doubt about the syntax
Description=HipHop Virtual Machine (FCGI)

ExecStartPre=/bin/mkdir -p /run/hhvm 
ExecStartPre=/bin/chown hhvm:hhvm /run/hhvm
ExecStart=/usr/bin/hhvm -c /etc/hhvm/server.ini -c /etc/hhvm/php.ini --user=hhvm --mode daemon [Install] WantedBy=multi-user.target 
d /run/hhvm 0755 hhvm hhvm -

Now the daemon started, but my webfront could not read its socket. This was where it would have been nice to be able to set the socket user and group, but alas, HHVM does not yet support that. There was already work towards that, but a large changeset into the whole FastCGI implementation got (quite understandably) the priority of way for being pulled in and no one ever got around to rebasing the configuration work.

I quite inelegantly solved this for now by simply letting hhvm run as the webfront user.

Also, HHVM caches code to disk, so after tinkering around a while I had multiple .hhvm.hhbc files littered around my system and was scratching my head a bit about why. It defaults to ~/.hhvm.hhbc. This is where the hint I mentioned earlier came in handy on figuring out what is up with that.

Initial impressions

  • It likes to use a lot of memory, but it does so in a single process
    • Quite nicely easy to track at a glance vs. older worker process models
    • Out of the box it uses the same as ~10 workers would use
      • I would not use so many workers for this deployment myself
      • It’ll be quite interesting to see how this lives over time and different loads
  • There is no way to meaningfully limit how much memory it can use
    • This blog will most likely outright crash my server in case of any serious traffic
    • The ugly hack would be to use systemd resource controls
      • Restart the service when a certain memory limit is hit
    • The real solution would be to have an infrastructure where you have dedicated HHVM boxes
  • It pretty much does what it says on the tin
    • It already supports most of what one would imagine to do with it