How to Deploy with Heroku

Starting with the release of RLetters 3.0, the application is optimized to be run on Heroku, with text content served from an Apache Solr instance on Amazon EC2. Read on to see how we set up our own instance, evoText.


Deploy a Solr instance in Amazon EC2

To get a Solr instance up and running, start by deploying an EC2 instance running Amazon’s basic Linux AMI. Then:

  1. Update packages and install Java 8:

    $ sudo yum upgrade
    $ sudo yum install java-1.8.0-openjdk-headless
    
  2. Download the Solr distribution for our current version, and install it as a global service:

    $ wget http://archive.apache.org/dist/lucene/solr/7.3.0/solr-7.3.0.tgz
    $ tar xf solr-7.3.0.tgz solr-7.3.0/bin/install_solr_service.sh --strip-components=2
    $ sudo bash ./install_solr_service.sh solr-7.3.0.tgz
    
  3. You need to configure some increased ulimits to make Solr happy. Open the file /etc/security/limits.d/99-solr.conf, and add:

    *       soft    nproc   65000
    *       hard    nproc   65000
    *       soft    nofile  65000
    *       hard    nofile  65000
    
  4. To set up the Solr configuration, you’ll need to create the Solr core, and to download the configuration from our sample Solr installation.

    $ wget https://github.com/rletters/solr-example/archive/master.zip
    $ unzip master.zip 'solr-example-master/server/solr/configsets/*' solr-example-master/server/solr/cores/core1/core.properties
    $ rm master.zip
        
    $ sudo mkdir -p /var/solr/data/cores/core1
    $ sudo mkdir /var/solr/data/configsets
        
    $ sudo mv -i solr-example-master/server/solr/configsets/rletters /var/solr/data/configsets/
    $ sudo mv -i solr-example-master/server/solr/cores/core1/core.properties /var/solr/data/cores/core1
    
    $ sudo chown -R solr:solr /var/solr/data
    
    $ rm -r solr-example-master
    
  5. Finally, restart Solr:

    $ sudo /etc/init.d/solr restart
    

Your Solr instance should now be up and running on port 8983. The next step is to secure it, so that it may be read from the web and worker dynos, but not written to from the wider internet.

Secure the Solr server

Enable SSL

First, it is strongly recommended that you configure SSL on the server. This will require giving it a stable domain name, as well as an Elastic IP address that will permanently map to the server on the public internet. See the AWS documentation on elastic IP addresses and DNS routing to EC2 instances for more information.

With that done, you can use Let’s Encrypt to generate certificates for your server. There is an Amazon documentation page on the topic here, and the short version looks like:

$ wget -r --no-parent -A 'epel-release-*.rpm' http://dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/
$ sudo rpm -Uvh dl.fedoraproject.org/pub/epel/7/x86_64/Packages/e/epel-release-*.rpm
$ sudo yum-config-manager --enable epel*
$ sudo yum repolist all
$ rm -r dl.fedoraproject.org

$ sudo yum install certbot

Run certbot to create your first certificate. Then, automate the renewal process as follows. Set up a cron job to run twice-daily, which executes the following script (as /usr/local/bin/renew-cert):

#!/bin/bash

certbot renew --no-self-upgrade --deploy-hook /usr/local/bin/copy-cert

And create the directory /etc/solr-keys and save the following script as /usr/local/bin/copy-cert:

#!/bin/bash
set -e

IN_DIR=/etc/letsencrypt/live/YOUR_DOMAIN_NAME
OUT_DIR=/etc/solr-keys

openssl pkcs12 -export -in "$IN_DIR/fullchain.pem" -inkey "$IN_DIR/privkey.pem" -out "$OUT_DIR/cert.p12" -passout pass:pkcs-12-pass

rm -f "$OUT_DIR/cert.jks"
keytool -importkeystore -srckeystore "$OUT_DIR/cert.p12" -srcstoretype PKCS12 -srcstorepass pkcs-12-pass -destkeystore "$OUT_DIR/cert.jks" -deststoretype JKS -deststorepass solr-keystore-pass -destkeypass solr-keystore-pass

Finally, configure the Solr server itself at /etc/default/solr.in.sh:

SOLR_SSL_ENABLED=true
# Uncomment to set SSL-related system properties
# Be sure to update the paths to the correct keystore for your environment
SOLR_SSL_KEY_STORE=/etc/solr-keys/cert.jks
SOLR_SSL_KEY_STORE_PASSWORD=solr-keystore-pass
SOLR_SSL_KEY_STORE_TYPE=JKS
SOLR_SSL_TRUST_STORE=/etc/solr-keys/cert.jks
SOLR_SSL_TRUST_STORE_PASSWORD=solr-keystore-pass
SOLR_SSL_TRUST_STORE_TYPE=JKS

Enable basic authentication

Generate a new password for your Solr installation, and enable authentication in the control script by uncommenting the following lines in /etc/default/solr.in.sh:

SOLR_AUTH_TYPE="basic"
SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:YOURPASSWORD"

Then, save the following as /var/solr/security.json:

{
"authentication":{ 
   "blockUnknown": true, 
   "class":"solr.BasicAuthPlugin",
   "credentials":{"solr":"IV0EHq1OnNrj6gvRCwvFwTrZ1+z1oBbnQdiVC3otuq0= Ndd7LKvVBAaZIF0QAVi1ekCfAJXr1GGfLtRUXhgrF8c="} 
},
"authorization":{
   "class":"solr.RuleBasedAuthorizationPlugin",
   "permissions":[{"name":"all",
      "role":"admin"}], 
   "user-role":{"solr":"admin"} 
}}

This creates a user with the username Solr and password ‘SolrRocks’. You can change that password by executing:

$ curl --user solr:SolrRocks https://localhost:8983/solr/admin/authentication -H 'Content-type:application/json' -d '{"set-user": {"solr":"n8C7CSkJT6NAG2zu39kJ85T3u8Gu8WgD"}}'

Load Data into Solr

You can now use the Solr post script, found at /opt/solr/bin/post, to post your article content to your Solr server. The schema for this content can be found in our example Solr repository, and an example document with all fields can be found there as well.


Set up a Heroku application

To begin, you can follow the tutorial present at Heroku to set up your local workstation. The abbreviated version will look something like this:

$ heroku login
$ git clone https://github.com/rletters/rletters.git
$ cd rletters
$ bundle
$ heroku create

Don’t push yet, as it won’t build until we configure some custom buildpacks.

Set buildpacks

Because RLetters uses Yarn with Rails, you will need to activate the Heroku buildpack for Node.js, in addition to the buildpack for Ruby. Run:

$ heroku buildpacks:add -i 1 heroku/nodejs
$ heroku buildpacks:add -i 2 heroku/ruby

Now, the application should be able to build, so run:

$ git push heroku master

Set application configuration

First, get a new set of secret keys for your application:

$ rails rletters:secrets:print
Set the following environment variables:
SECRET_KEY_BASE=1234...
DEVISE_SECRET_KEY=1234...

Then you’ll want to set, at least, the following configuration values:

$ heroku config:set \
    ADMIN_PASSWORD=anewadminpassword \
    SOLR_URL=https://solr:YOURPASSWORD@YOURDOMAIN:8983/solr/core1 \
    SOLR_TIMEOUT=120 \
    SECRET_KEY_BASE=1234... \
    DEVISE_SECRET_KEY=1234... \
    S3_ACCESS_KEY_ID=asdf... \
    S3_SECRET_ACCESS_KEY=asdf... \
    S3_BUCKET=bucket_name \
    S3_REGION=us-east-1

Seed the database

To get the DB up and running, execute:

$ heroku run rails db:schema:load db:seed

If you already have an RLetters instance and are migrating, you can run the database migrations with:

$ heroku run rails db:migrate

Note that there is no automatic migration path from RLetters 2.0 to RLetters 3.0.

Activate scheduled tasks

Two tasks require being run by the Heroku scheduler. Activate it and open its dashboard:

$ heroku addons:create scheduler:standard
$ heroku addons:open scheduler

Add two tasks:

  1. Command: bin/rails rletters:maintenance:expire_tasks
    Frequency: Daily
    Choose a reasonable run time for your timezone and user base

  2. Command: bin/rails rletters:jobs:maintenance
    Frequency: Every ten minutes

These two tasks will take care of cleaning up old user jobs and running maintenance-queue tasks (such as sending emails and maintaining the ActiveStorage file database).

If you have a particularly high-traffic RLetters instance, you will need to run the maintenance queue as a worker process, not as a scheduled job. You may edit the Procfile to accomplish this.

Activate autoscaling

There are two choices for running your worker jobs on Heroku. You can either leave a worker box running 24/7 on your analysis queue by spinning up a permanent worker instance, or you can configure automatic scaling. First, generate an API key:

$ heroku plugins:install heroku-cli-oauth
$ heroku authorizations:create -d "Autoscaling API token"
Created OAuth authorization.
  ID:          2f01aac0-e9d3-4773-af4e-3e510aa006ca
  Description: Autoscaling API token
  Scope:       global
  Token:       e7dd6ad7-3c6a-411e-a2be-c9fe52ac7ed2

Then, enable dyno metadata for your app (required so that our code knows what Heroku application you’re running):

$ heroku labs:enable runtime-dyno-metadata

Finally, set the API key as a configuration variable and enable autoscaling:

$ heroku config:set \
    AUTOSCALE_API_KEY=2f01aac0-e9d3-4773-af4e-3e510aa006ca \
    AUTOSCALE_JOBS=heroku

Now, the RLetters code will automatically hire a Heroku worker for analysis jobs when the queue is full, and fire it when the queue is empty.

All content copyright © 2010–2018 Charles Pence. All web content is released under CC-BY-NC-SA 3.0.
Code is released under the MIT License. RLetters logo taken from a photograph by Leo Reynolds.