View source for 2018/01/14

==2018-01-14==
My current plan is:
# create new system user "tootcat"
# create a new Postgres db for toot.cat's Mastodon
# migrate the data there
# install Mastodon under user "tootcat"
# configure nginx
# run Mastodon's schema upgrade rake task
# test the result as https://new.toot.cat (get it working)
# write a script to automate the data migration
# in close succession:
## run the migration script
## reconfigure nginx to point to the new instance

====revisions====
* Decided it would simplify things if I went ahead and created the "tootcat" system user first.
* [https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Production-guide.md Instructions] wanted to configure nginx after installing Mastodon, and this did need to be looked into at some point, so added it as step 5.
===Notes===
Upgrading Mastodon: 
===Work Log===
====step 1====
Created user and group "tootcat" with /bin/bash as shell, no login allowed. I checked user "polymerwitch" (user for ''current'' Mastodon instance), and it has "sudo" membership -- but I'm not adding that, because it shouldn't be needed.
====step 2====
I'm calling the new db <code>tootcat-masto-r1</code> (toot.cat Mastodon db revision 1).
: <code>root@tootcat:~# '''createdb --owner=tootcat tootcat-masto-r1'''</code>
:: <code>createdb: could not connect to database template1: FATAL:  role "root" does not exist</code>
::: Apparently pg assumes whatever the current system user is as its internal user as well.
: <code>root@tootcat:~# '''su - postgres -c 'createdb --owner=tootcat tootcat-masto-r1''''</code>
:: <code>createdb: database creation failed: ERROR:  role "tootcat" does not exist</code>
: <code>root@tootcat:~# '''createuser --createdb tootcat'''</code>
:: <code>createuser: could not connect to database postgres: FATAL:  role "root" does not exist</code> - can't say I'm surprised
: <code>root@tootcat:~# '''su - postgres -c 'createuser --createdb tootcat''''</code>
:: Success.
: <code>root@tootcat:~# '''su - postgres -c 'createdb --owner=tootcat tootcat-masto-r1''''</code>
:: Success.
====step 3: migrate the data====
Done earlier -- inside <code>/root/backups</code>:
: <code>su - postgres -c 'pg_dump mastodon' > tootcat.sql</code>

[https://www.postgresql.org/docs/current/static/backup-dump.html Importing the data dump] into the new db:
: <code>root@tootcat:~# '''cd /root/scratch'''</code>
: <code>root@tootcat:~/scratch# '''ls'''</code>
:: <code>tootcat.sql</code>
: <code>root@tootcat:~/scratch# '''psql tootcat-masto-r1 < tootcat.sql'''</code>
:: <code>psql: FATAL:  role "root" does not exist</code>
::: I actually expected this, but just wanted to make sure.
: <code>root@tootcat:~/scratch# '''su - postgres -c 'psql tootcat-masto-r1 < tootcat.sql'''</code>
:: <code>-su: tootcat.sql: No such file or directory</code>
::: Apparently this operates relative to the <code>postgres</code> user's home folder, or wherever you get put by default when logging in. Okay.
: <code>root@tootcat:~/scratch# '''su - postgres -c 'psql tootcat-masto-r1 < /root/scratch/tootcat.sql''''</code>
:: <code>-su: /root/scratch/tootcat.sql: Permission denied</code>
::: This is also unsurprising. Other users shouldn't have access to /root files.

It was more or less at this point that I discovered that the <code>tootcat.sql</code> file was zero bytes long, so apparently the export process wasn't happy either.

There's no <code>/home/postgres</code>, so we can't go there.

<code>/root/scratch</code> is already mode 777, so apparently that's insufficient.

Oh, but the ''actual'' <code>tootcat.sql</code> file is in <code>/root/backups</code>. Just to be sure, then:

: <code>root@tootcat:~/scratch# '''cd ../backups/'''</code>
: <code>root@tootcat:~/backups# '''su - postgres -c 'psql tootcat-masto-r1 < /root/backups/tootcat.sql''''</code>
:: <code>-su: /root/backups/tootcat.sql: Permission denied</code>
::: As expected.

And then it finally occurred to me:
: <code>root@tootcat:~/backups# '''su - postgres -c 'pwd''''</code>
:: <code>/var/lib/postgresql</code>
::: Oh! Sneaksy, having a home folder outside of the <code>/home</code> folder. Just because <code>/root</code> does it doesn't make it okay; do as I say, not as I do....
: <code>root@tootcat:~/backups# '''cd /var/lib/postgresql/'''</code>
: <code>root@tootcat:/var/lib/postgresql# '''mv /root/backups/tootcat.sql ./'''</code>
: <code>root@tootcat:/var/lib/postgresql# '''chown postgres:postgres tootcat.sql'''</code>
: <code>root@tootcat:/var/lib/postgresql# '''su - postgres -c 'psql tootcat-masto-r1 < tootcat.sql''''</code>
:: ...and then stuff started happening that ''might'' have been the output of a successful import.

After taking a brief break, I come back to find that the process has completed... and a brief inspection shows that the db seems to be populated with data. I'm calling this step complete, then.
====step 4: installing Mastodon====
The [https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Updating-Mastodon-Guide.md official upgrade instructions] mention that you should only ever use "tagged releases on production Mastodon instances", and therefore:
<syntaxhighlight lang=bash>
su - mastodon
cd ~/live
git pull
git checkout $(git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)
</syntaxhighlight>
...except corrected for how things are set up here... and also I need to <code>git clone</code> instead of <code>git pull</code> since I'm doing the upgrade on a fresh deployment rather than in place on an existing one.
: {{markup/tty|root@tootcat:/var/lib/postgresql#| cd /home/tootcat}}
: {{markup/tty|root@tootcat:/home/tootcat#| su - tootcat}}
: {{markup/tty|tootcat@tootcat:~$| mkdir web}}
: {{markup/tty|tootcat@tootcat:~$| cd web}}
: {{markup/tty|tootcat@tootcat:~/web$| mkdir mastodon}}
: {{markup/tty|tootcat@tootcat:~/web$| cd mastodon}}
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| git clone https://github.com/rbenv/rbenv.git ~/.rbenv}}
:: {{markup/tty|Cloning into '/home/tootcat/.rbenv'...}}
:: {{markup/tty|remote: Counting objects: 2706, done.}}
:: {{markup/tty|remote: Total 2706 (delta 0), reused 0 (delta 0), pack-reused 2706}}
:: {{markup/tty|Receiving objects: 100% (2706/2706), 503.75 KiB | 0 bytes/s, done.}}
:: {{markup/tty|Resolving deltas: 100% (1695/1695), done.}}
:: {{markup/tty|Checking connectivity... done.}}
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| cd ~/.rbenv && src/configure && make -C src}}
:: {{markup/tty|make: Entering directory '/home/tootcat/.rbenv/src'}}
:: {{markup/tty|gcc -fPIC     -c -o realpath.o realpath.c}}
:: {{markup/tty|gcc -shared -Wl,-soname,../libexec/rbenv-realpath.dylib  -o ../libexec/rbenv-realpath.dylib realpath.o }}
:: {{markup/tty|make: Leaving directory '/home/tootcat/.rbenv/src'}}
: {{markup/tty|tootcat@tootcat:~/.rbenv$| echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc}}
: {{markup/tty|tootcat@tootcat:~/.rbenv$ echo 'eval "$(rbenv init -)"' >> ~/.bashrc}}
: {{markup/tty|tootcat@tootcat:~/.rbenv$ exec bash}}
: {{markup/tty|tootcat@tootcat:~/.rbenv$ type rbenv}}
:: {{markup/tty|rbenv is a function}}
<syntaxhighlight lang=ruby>rbenv () 
{ 
    local command;
    command="${1:-}";
    if [ "$#" -gt 0 ]; then
        shift;
    fi;
    case "$command" in 
        rehash | shell)
            eval "$(rbenv "sh-$command" "$@")"
        ;;
        *)
            command rbenv "$command" "$@"
        ;;
    esac
}</syntaxhighlight>
: {{markup/tty|tootcat@tootcat:~/.rbenv$| git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build}}
:: {{markup/tty|Cloning into '/home/tootcat/.rbenv/plugins/ruby-build'...}}
:: {{markup/tty|remote: Counting objects: 8588, done.}}
:: {{markup/tty|remote: Compressing objects: 100% (25/25), done.}}
:: {{markup/tty|remote: Total 8588 (delta 9), reused 24 (delta 3), pack-reused 8556}}
:: {{markup/tty|Receiving objects: 100% (8588/8588), 1.79 MiB | 0 bytes/s, done.}}
:: {{markup/tty|Resolving deltas: 100% (5481/5481), done.}}
:: {{markup/tty|Checking connectivity... done.}}

Skipping the bit about installing Ruby, since that is already installed on this server.

chdir to <code>/home/tootcat/web</code>, then:
: {{markup/tty|tootcat@tootcat:~/web$| git clone https://github.com/tootsuite/mastodon.git mastodon}}
:: {{markup/tty|Cloning into 'mastodon'...}}
:: {{markup/tty|remote: Counting objects: 56389, done.}}
:: {{markup/tty|remote: Compressing objects: 100% (13/13), done.}}
:: {{markup/tty|remote: Total 56389 (delta 0), reused 3 (delta 0), pack-reused 56376}}
:: {{markup/tty|Receiving objects: 100% (56389/56389), 52.51 MiB | 30.41 MiB/s, done.}}
:: {{markup/tty|Resolving deltas: 100% (37140/37140), done.}}
:: {{markup/tty|Checking connectivity... done.}}
: {{markup/tty|tootcat@tootcat:~/web$| cd mastodon/}}
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| git checkout $(git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1)}}
:: {{markup/tty|Note: checking out 'v2.1.3'.}}
::
<pre>You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 35b8498... Skip ActivityPub Announces of non-public objects (#6230)</pre>
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| gem install bundler}}
:: {{markup/tty|Fetching: bundler-1.16.1.gem (100%)}}
:: {{markup/tty|ERROR:  While executing gem ... (Gem::FilePermissionError)}}
:: {{markup/tty|    You don't have write permissions for the /var/lib/gems/2.3.0 directory.}}
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| bundle exec rails c}}
:: {{markup/tty/color|red|Could not find rake-12.3.0 in any of the sources}}
:: {{markup/tty/color|orange|Run `bundle install` to install missing gems.}}
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| bundle install}}
:: {{markup/tty/color|orange|Warning: the running version of Bundler (1.15.3) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`.}}
:: {{markup/tty|Fetching gem metadata from https://rubygems.org/...........}}
:: {{markup/tty|Fetching version metadata from https://rubygems.org/...}}
:: {{markup/tty|Fetching dependency metadata from https://rubygems.org/..}}
:: {{markup/tty|Fetching rake 12.3.0}}
<pre>

Your user account isn't allowed to install to the system RubyGems.
  You can cancel this installation and run:

      bundle install --path vendor/bundle

  to install the gems into ./vendor/bundle/, or you can enter your password
  and install the bundled gems to RubyGems using sudo.

  Password: 
</pre>
Decided this was not the way to do it, and logged in as root:
: {{markup/tty|root@tootcat:~#| gem install bundler}}
:: {{markup/tty|Fetching: bundler-1.16.1.gem (100%)}}
:: {{markup/tty|Successfully installed bundler-1.16.1}}
:: {{markup/tty|Parsing documentation for bundler-1.16.1}}
:: {{markup/tty|Installing ri documentation for bundler-1.16.1}}
:: {{markup/tty|Done installing documentation for bundler after 9 seconds}}
:: {{markup/tty|1 gem installed}}
: {{markup/tty|root@tootcat:~#| bundle install --deployment --without development test}}
:: {{markup/tty/color|orange|Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine.}}
:: {{markup/tty/color|red|Could not locate Gemfile}}
Okay, so I guess it's back to the mastodon user...
: {{markup/tty|tootcat@tootcat:~/web/mastodon$| bundle install --deployment --without development test}}
:: This seemed to work: [[/paste/2018/01/14/01]]
: {{markup/tty|tootcat@tootcat:~/web/mastodon$ yarn install --pure-lockfile}}
:: This also seemed successful, but it also asked me to upgrade Yarn with these specific commands:
: {{markup/tty|tootcat@tootcat:~/web/mastodon$ npm upgrade --global yarn}}
: {{markup/tty|tootcat@tootcat:~/web/mastodon$ npm rebuild node-sass}}
<pre>
> node-sass@4.7.2 install /home/tootcat/web/mastodon/node_modules/node-sass
> node scripts/install.js

node-sass build Binary found at /home/tootcat/web/mastodon/node_modules/node-sass/vendor/linux-x64-57/binding.node

> node-sass@4.7.2 postinstall /home/tootcat/web/mastodon/node_modules/node-sass
> node scripts/build.js

Binary found at /home/tootcat/web/mastodon/node_modules/node-sass/vendor/linux-x64-57/binding.node
Testing binary
Binary is fine
node-sass@4.7.2 /home/tootcat/web/mastodon/node_modules/node-sass
</pre>
...and then we move on to configuring Postgres (already done) and nginx, which I think I should add as step 5.
===step 5: configure nginx - ABANDONED for now===
I made a copy of /etc/nginx/sites-available/default and called it toot-cat-new, then changed just a couple of things:
* The '''server_name''' was changed from <code>toot.cat</code> to <code>new.toot.cat</code> in two places.

...and then I realized that this plan was going to require setting up extra instances of Puma and the media server (name?) on different ports from 3000 and 4000... which I think I know how to do, but this kinda crosses a threshold into territory uncertain enough that I decided to let a professional do it this time, and live to fight another day.