2018/01/14

Sunday, January 14, 2018 (#14)

Saturday Sunday Monday [no prev] .. today .. next 2024-06-29 (Sat) 20:06 EST Migration complete? Some things are being a bit odd, but hopefully they'll settle down... There are no known issues. 2018-01-14 My current plan is: create new system user "tootcat" create a new Postgres db for toot.cat's Mastodon migrate the data there install Mastodon under user "tootcat" configure nginx run Mastodon's schema upgrade rake task test the result as https://new.toot.cat (get it working) write a script to automate the data migration in close succession: run the migration script reconfigure nginx to point to the new instance revisions Decided it would simplify things if I went ahead and created the "tootcat" system user first. Instructions wanted to configure nginx after installing Mastodon, and this did need to be looked into at some point, so added it as step 5. Notes Upgrading Mastodon: Work Log step 1 Created user and group "tootcat" with /bin/bash as shell, no login allowed. I checked user "polymerwitch" (user for current Mastodon instance), and it has "sudo" membership -- but I'm not adding that, because it shouldn't be needed. step 2 I'm calling the new db tootcat-masto-r1 (toot.cat Mastodon db revision 1). root@tootcat:~# createdb --owner=tootcat tootcat-masto-r1 createdb: could not connect to database template1: FATAL: role "root" does not exist Apparently pg assumes whatever the current system user is as its internal user as well. root@tootcat:~# su - postgres -c 'createdb --owner=tootcat tootcat-masto-r1' createdb: database creation failed: ERROR: role "tootcat" does not exist root@tootcat:~# createuser --createdb tootcat createuser: could not connect to database postgres: FATAL: role "root" does not exist - can't say I'm surprised root@tootcat:~# su - postgres -c 'createuser --createdb tootcat' Success. root@tootcat:~# su - postgres -c 'createdb --owner=tootcat tootcat-masto-r1' Success. step 3: migrate the data Done earlier -- inside /root/backups: su - postgres -c 'pg_dump mastodon' > tootcat.sql Importing the data dump into the new db: root@tootcat:~# cd /root/scratch root@tootcat:~/scratch# ls tootcat.sql root@tootcat:~/scratch# psql tootcat-masto-r1 < tootcat.sql psql: FATAL: role "root" does not exist I actually expected this, but just wanted to make sure. root@tootcat:~/scratch# su - postgres -c 'psql tootcat-masto-r1 < tootcat.sql -su: tootcat.sql: No such file or directory Apparently this operates relative to the postgres user's home folder, or wherever you get put by default when logging in. Okay. root@tootcat:~/scratch# su - postgres -c 'psql tootcat-masto-r1 < /root/scratch/tootcat.sql' -su: /root/scratch/tootcat.sql: Permission denied This is also unsurprising. Other users shouldn't have access to /root files. It was more or less at this point that I discovered that the tootcat.sql file was zero bytes long, so apparently the export process wasn't happy either. There's no /home/postgres, so we can't go there. /root/scratch is already mode 777, so apparently that's insufficient. Oh, but the actual tootcat.sql file is in /root/backups. Just to be sure, then: root@tootcat:~/scratch# cd ../backups/ root@tootcat:~/backups# su - postgres -c 'psql tootcat-masto-r1 < /root/backups/tootcat.sql' -su: /root/backups/tootcat.sql: Permission denied As expected. And then it finally occurred to me: root@tootcat:~/backups# su - postgres -c 'pwd' /var/lib/postgresql Oh! Sneaksy, having a home folder outside of the /home folder. Just because /root does it doesn't make it okay; do as I say, not as I do.... root@tootcat:~/backups# cd /var/lib/postgresql/ root@tootcat:/var/lib/postgresql# mv /root/backups/tootcat.sql ./ root@tootcat:/var/lib/postgresql# chown postgres:postgres tootcat.sql root@tootcat:/var/lib/postgresql# su - postgres -c 'psql tootcat-masto-r1 < tootcat.sql' ...and then stuff started happening that might have been the output of a successful import. After taking a brief break, I come back to find that the process has completed... and a brief inspection shows that the db seems to be populated with data. I'm calling this step complete, then. step 4: installing Mastodon The official upgrade instructions mention that you should only ever use "tagged releases on production Mastodon instances", and therefore: su - mastodon cd ~/live git pull git checkout $(git tag -l | grep -v 'rc[0-9]*$' | sort -V | tail -n 1) ...except corrected for how things are set up here... and also I need to git clone instead of git pull since I'm doing the upgrade on a fresh deployment rather than in place on an existing one. root@tootcat:/var/lib/postgresql# cd /home/tootcat root@tootcat:/home/tootcat# su - tootcat tootcat@tootcat:~$ mkdir web tootcat@tootcat:~$ cd web tootcat@tootcat:~/web$ mkdir mastodon tootcat@tootcat:~/web$ cd mastodon tootcat@tootcat:~/web/mastodon$ git clone https://github.com/rbenv/rbenv.git ~/.rbenv Cloning into '/home/tootcat/.rbenv'... remote: Counting objects: 2706, done. remote: Total 2706 (delta 0), reused 0 (delta 0), pack-reused 2706 Receiving objects: 100% (2706/2706), 503.75 KiB 0 bytes/s, done. Resolving deltas: 100% (1695/1695), done. Checking connectivity... done. tootcat@tootcat:~/web/mastodon$ cd ~/.rbenv && src/configure && make -C src make: Entering directory '/home/tootcat/.rbenv/src' gcc -fPIC -c -o realpath.o realpath.c gcc -shared -Wl,-soname,../libexec/rbenv-realpath.dylib -o ../libexec/rbenv-realpath.dylib realpath.o make: Leaving directory '/home/tootcat/.rbenv/src' tootcat@tootcat:~/.rbenv$ tootcat@tootcat:~/.rbenv$ echo 'eval "$(rbenv init -)"' >> ~/.bashrc tootcat@tootcat:~/.rbenv$ exec bash tootcat@tootcat:~/.rbenv$ type rbenv rbenv is a function rbenv () { local command; command="${1:-}"; if [ "$#" -gt 0 ]; then shift; fi; case "$command" in rehash | shell) eval "$(rbenv "sh-$command" "$@")" ;; *) command rbenv "$command" "$@" ;; esac } tootcat@tootcat:~/.rbenv$ git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build Cloning into '/home/tootcat/.rbenv/plugins/ruby-build'... remote: Counting objects: 8588, done. remote: Compressing objects: 100% (25/25), done. remote: Total 8588 (delta 9), reused 24 (delta 3), pack-reused 8556 Receiving objects: 100% (8588/8588), 1.79 MiB 0 bytes/s, done. Resolving deltas: 100% (5481/5481), done. Checking connectivity... done. Skipping the bit about installing Ruby, since that is already installed on this server. chdir to /home/tootcat/web, then: tootcat@tootcat:~/web$ git clone https://github.com/tootsuite/mastodon.git mastodon Cloning into 'mastodon'... remote: Counting objects: 56389, done. remote: Compressing objects: 100% (13/13), done. remote: Total 56389 (delta 0), reused 3 (delta 0), pack-reused 56376 Receiving objects: 100% (56389/56389), 52.51 MiB 30.41 MiB/s, done. Resolving deltas: 100% (37140/37140), done. Checking connectivity... done. tootcat@tootcat:~/web$ cd mastodon/ tootcat@tootcat:~/web/mastodon$ git checkout $(git tag -l Note: checking out 'v2.1.3'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by performing another checkout. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -b with the checkout command again. Example: git checkout -b <new-branch-name> HEAD is now at 35b8498... Skip ActivityPub Announces of non-public objects (#6230) tootcat@tootcat:~/web/mastodon$ gem install bundler Fetching: bundler-1.16.1.gem (100%) ERROR: While executing gem ... (Gem::FilePermissionError) You don't have write permissions for the /var/lib/gems/2.3.0 directory. tootcat@tootcat:~/web/mastodon$ bundle exec rails c Could not find rake-12.3.0 in any of the sources Run `bundle install` to install missing gems. tootcat@tootcat:~/web/mastodon$ bundle install Warning: the running version of Bundler (1.15.3) is older than the version that created the lockfile (1.16.1). We suggest you upgrade to the latest version of Bundler by running `gem install bundler`. Fetching gem metadata from https://rubygems.org/........... Fetching version metadata from https://rubygems.org/... Fetching dependency metadata from https://rubygems.org/.. Fetching rake 12.3.0 Your user account isn't allowed to install to the system RubyGems. You can cancel this installation and run: bundle install --path vendor/bundle to install the gems into ./vendor/bundle/, or you can enter your password and install the bundled gems to RubyGems using sudo. Password: Decided this was not the way to do it, and logged in as root: root@tootcat:~# gem install bundler Fetching: bundler-1.16.1.gem (100%) Successfully installed bundler-1.16.1 Parsing documentation for bundler-1.16.1 Installing ri documentation for bundler-1.16.1 Done installing documentation for bundler after 9 seconds 1 gem installed root@tootcat:~# bundle install --deployment --without development test Don't run Bundler as root. Bundler can ask for sudo if it is needed, and installing your bundle as root will break this application for all non-root users on this machine. Could not locate Gemfile Okay, so I guess it's back to the mastodon user... tootcat@tootcat:~/web/mastodon$ bundle install --deployment --without development test This seemed to work: /paste/2018/01/14/01 tootcat@tootcat:~/web/mastodon$ yarn install --pure-lockfile This also seemed successful, but it also asked me to upgrade Yarn with these specific commands: tootcat@tootcat:~/web/mastodon$ npm upgrade --global yarn tootcat@tootcat:~/web/mastodon$ npm rebuild node-sass > node-sass@4.7.2 install /home/tootcat/web/mastodon/node_modules/node-sass > node scripts/install.js node-sass build Binary found at /home/tootcat/web/mastodon/node_modules/node-sass/vendor/linux-x64-57/binding.node > node-sass@4.7.2 postinstall /home/tootcat/web/mastodon/node_modules/node-sass > node scripts/build.js Binary found at /home/tootcat/web/mastodon/node_modules/node-sass/vendor/linux-x64-57/binding.node Testing binary Binary is fine node-sass@4.7.2 /home/tootcat/web/mastodon/node_modules/node-sass ...and then we move on to configuring Postgres (already done) and nginx, which I think I should add as step 5. step 5: configure nginx - ABANDONED for now I made a copy of /etc/nginx/sites-available/default and called it toot-cat-new, then changed just a couple of things: The server_name was changed from toot.cat to new.toot.cat in two places. ...and then I realized that this plan was going to require setting up extra instances of Puma and the media server (name?) on different ports from 3000 and 4000... which I think I know how to do, but this kinda crosses a threshold into territory uncertain enough that I decided to let a professional do it this time, and live to fight another day.