2018/04/07

Saturday, April 7, 2018 (#97)

Friday Saturday Sunday prev .. today .. next 2024-06-29 (Sat) 20:06 EST Migration complete? Some things are being a bit odd, but hopefully they'll settle down... There are no known issues. Take 1 Loosely following these instructions... I just do everything in root rather than using sudo, because that's how I roll. Things done: Create droplet (4GB- 2CPU - 80GB - $20, Ubuntu 16.04, SFO2, IPv6, Gonzo key) apt update apt upgrade install node.js: curl -sL https://deb.nodesource.com/setup_6.x | sudo bash - apt install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev nodejs npm install -g yarn install redis-server: apt install redis-server redis-tools install PostgreSQL (Pg): apt install postgresql postgresql-contrib switch to postgres user, create Mastodon user in Pg: su - postgres psql CREATE USER mastodon CREATEDB; \q Now, we're trying to enable ident auth in Pg so users can log in without a password. I don't understand why we want to do this; seems like bad security. I tried to do this bit exactly as written, because it looked like it might be user-sensitive, but it didn't work (see below). Can this be done as all one line? sudo sed -i '/^local.*postgres.*peer$/a host all all 127.0.0.1/32 ident' \ /etc/postgresql/9.?/main/pg_hba.conf That didn't work, because after the second line it asked for a password for postgres, which I don't have, and wouldn't accept a blank. So, backing out and trying as root (and all one line): sed -i '/^local.*postgres.*peer$/a host all all 127.0.0.1/32 ident' /etc/postgresql/9.?/main/pg_hba.conf No error, but also no success message. I will assume successful. install ident daemon: apt install pidentd systemctl enable pidentd Response: pidentd.service is not a native service, redirecting to systemd-sysv-install Executing /lib/systemd/systemd-sysv-install enable pidentd systemctl start pidentd systemctl restart postgresql Setting Up Ruby apt install autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev rbenv Note: the instructions omitted rbenv, but it seems to be needed. adduser --disabled-password --disabled-login mastodon Just hitting ENTER in response to all questions, "Y" in response to "Y/n". Change to user mastodon (which also puts you in /home/mastodon): su - mastodon git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build install Ruby 2.4.1: rbenv install 2.4.1 That didn't work because rbenv was not installed. (Also not sure this is the best version of Ruby, anymore, but that seems fixable later now that we have the command). So I had to back out to root, apt install rbenv, and then su - mastodon again. rbenv install 2.4.1 This takes a couple of minutes. rbenv global 2.4.1 This is instantaneous. ruby -v response: ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu] ...um, okay. Installing Mastodon cd ~ This is probably redundant. git clone https://github.com/tootsuite/mastodon.git live cd live git checkout $(git tag | tail -n 1) I don't know what this does. "Install bundler to manage the dependencies and disable the gem documentation." echo "gem: --no-document" > ~/.gemrc gem install bundler --no-ri Response: Fetching: bundler-1.16.1.gem (100%) ERROR: While executing gem ... (Gem::FilePermissionError) You don't have write permissions for the /var/lib/gems/2.3.0 directory. Okay, Simon says... exit gem install bundler --no-ri Response: Fetching: bundler-1.16.1.gem (100%) Successfully installed bundler-1.16.1 1 gem installed su - mastodon cd live bundle install --deployment --without development test A bunch of stuff happened, and then this error message: Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /home/mastodon/live/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri /usr/bin/ruby2.3 -r ./siteconf20180407-2653-idvqpi.rb extconf.rb mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h extconf failed, exit code 1 Gem files will remain installed in /home/mastodon/live/vendor/bundle/ruby/2.3.0/gems/nokogiri-1.8.2 for inspection. Results logged to /home/mastodon/live/vendor/bundle/ruby/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out An error occurred while installing nokogiri (1.8.2), and Bundler cannot continue. Make sure that `gem install nokogiri -v '1.8.2'` succeeds before bundling. In Gemfile: rails-settings-cached was resolved to 0.6.6, which depends on rails was resolved to 5.1.4, which depends on actioncable was resolved to 5.1.4, which depends on actionpack was resolved to 5.1.4, which depends on actionview was resolved to 5.1.4, which depends on rails-dom-testing was resolved to 2.0.3, which depends on nokogiri Okay, trying that command... gem install nokogiri -v '1.8.2' Response: Fetching: mini_portile2-2.3.0.gem (100%) ERROR: While executing gem ... (Gem::FilePermissionError) You don't have write permissions for the /var/lib/gems/2.3.0 directory. So again, Simon says... exit gem install nokogiri -v '1.8.2' Response: Fetching: mini_portile2-2.3.0.gem (100%) Successfully installed mini_portile2-2.3.0 Fetching: nokogiri-1.8.2.gem (100%) Building native extensions. This could take a while... ERROR: Error installing nokogiri: ERROR: Failed to build gem native extension. current directory: /var/lib/gems/2.3.0/gems/nokogiri-1.8.2/ext/nokogiri /usr/bin/ruby2.3 -r ./siteconf20180407-2736-1uhzyd6.rb extconf.rb mkmf.rb can't find header files for ruby at /usr/lib/ruby/include/ruby.h extconf failed, exit code 1 Gem files will remain installed in /var/lib/gems/2.3.0/gems/nokogiri-1.8.2 for inspection. Results logged to /var/lib/gems/2.3.0/extensions/x86_64-linux/2.3.0/nokogiri-1.8.2/gem_make.out Okay. Let's start over, and use the official instructions this time, leaving off any bits that have already clearly been done. (I don't know why I didn't use them to begin with... I guess these alternate instructions just looked so distro-specific and friendly.) Take 2 As root... cd /root mkdir setup cd setup wget https://deb.nodesource.com/setup_6.x ...and then "review the script" using less -- as if I'd know how to evaluate it. Glanced at it, what does it mean. Now running it... bash setup_6.x Yarn Repository apt -y install curl was already installed, but might not have been if I hadn't tried the other instructions first curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list apt update Install Yarn plus a bunch of other stuff: apt -y install imagemagick ffmpeg libpq-dev libxml2-dev libxslt1-dev file git g++ libprotobuf-dev protobuf-compiler pkg-config nodejs gcc autoconf bison build-essential libssl-dev libyaml-dev libreadline6-dev zlib1g-dev libncurses5-dev libffi-dev libgdbm3 libgdbm-dev nginx redis-server redis-tools postgresql postgresql-contrib nginx letsencrypt yarn User-Specific Stuff su - mastodon git clone https://github.com/rbenv/rbenv.git ~/.rbenv This does not seem to have been done earlier, or at least not with the same command -- but there is a .rbenv folder. So can't do it again, unless we want to erase it. Maybe later, if something seems to be wrong with rbenv. cd ~/.rbenv && src/configure && make -C src Response: -su: src/configure: No such file or directory Okay, something seems to be wrong with rbenv. cd ~ rm -rdf .rbenv git clone https://github.com/rbenv/rbenv.git ~/.rbenv cd ~/.rbenv && src/configure && make -C src echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc echo 'eval "$(rbenv init -)"' >> ~/.bashrc Restart the shell: exec bash Check if rbenv is correctly installed: type rbenv Response: rbenv is a function rbenv () { local command; command="${1:-}"; if [ "$#" -gt 0 ]; then shift; fi; case "$command" in rehash | shell) eval "$(rbenv "sh-$command" "$@")" ;; *) command rbenv "$command" "$@" ;; esac } Install ruby-build as rbenv plugin: git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build rbenv install 2.4.1 This is probably redundant now, but why not. It still takes a couple of minutes, though. rbenv global 2.4.1 Installing Mastodon Much of this has already been done -- but this instruction set does explain it a bit. Skipping redundant bits... Install bundler (again, but without the options): gem install bundler Use bundler to install the rest of the Ruby dependencies (this might be redundant, but maybe not): bundle install --deployment --without development test Response: Could not locate Gemfile I have to stop for tonight. part 2 Looks like Gemfile is an app-config file which needs to already exist when you run bundle install [...] -- and there's one in /home/mastodon/live, so let's run it from there. Log in as root, then: su - mastodon cd live bundle install --deployment --without development test Response: rbenv: version `2.5.0' is not installed (set by /home/mastodon/live/.ruby-version) Looks like the instructions may not have been updated; Charlotte tells me that Masto does, in fact, now require Ruby 2.5.0. So... cd ~ rbenv install 2.5.0 rbenv global 2.5.0 cd live bundle install --deployment --without development test Response: rbenv: bundle: command not found The `bundle' command exists in these Ruby versions: 2.4.1 I'm going on the assumption that this doesn't mean "Ruby did away with Bundler after 2.4.1" but rather "You haven't installed it for 2.5.0, you've only installed it for 2.4.1". We're already in ~/live, which seems to be the right folder, so no need to cd there. gem install bundler Successfully installed bundler-1.16.1 1 gem installed bundle install --deployment --without development test This seemed to be running successfully for quite awhile, installing various things, then failed with the following message: Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0/ext /home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-7030-grmz7k.rb extconf.rb checking for -lidn... no ERROR: could not find idn library! Please install the GNU IDN library or alternatively specify at least one of the following options if the library can only be found in a non-standard location: --with-idn-dir=/path/to/non/standard/location or --with-idn-lib=/path/to/non/standard/location/lib --with-idn-include=/path/to/non/standard/location/include *** extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/home/mastodon/.rbenv/versions/2.5.0/bin/$(RUBY_BASE_NAME) --with-idn-dir --without-idn-dir --with-idn-include --without-idn-include=${idn-dir}/include --with-idn-lib --without-idn-lib=${idn-dir}/lib --with-idnlib --without-idnlib To see why this extension failed to compile, please check the mkmf.log which can be found here: /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/mkmf.log extconf failed, exit code 1 Gem files will remain installed in /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0 for inspection. Results logged to /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/gem_make.out An error occurred while installing idn-ruby (0.1.0), and Bundler cannot continue. Make sure that `gem install idn-ruby -v '0.1.0'` succeeds before bundling. In Gemfile: idn-ruby Get back out to root, then: apt install idn su - mastodon cd live bundle install --deployment --without development test Response: Installing idn-ruby 0.1.0 with native extensions Gem::Ext::BuildError: ERROR: Failed to build gem native extension. current directory: /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0/ext /home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-19186-1hcp4ek.rb extconf.rb checking for -lidn... no ERROR: could not find idn library! Please install the GNU IDN library or alternatively specify at least one of the following options if the library can only be found in a non-standard location: --with-idn-dir=/path/to/non/standard/location or --with-idn-lib=/path/to/non/standard/location/lib --with-idn-include=/path/to/non/standard/location/include extconf.rb failed *** Could not create Makefile due to some reason, probably lack of necessary libraries and/or headers. Check the mkmf.log file for more details. You may need configuration options. Provided configuration options: --with-opt-dir --without-opt-dir --with-opt-include --without-opt-include=${opt-dir}/include --with-opt-lib --without-opt-lib=${opt-dir}/lib --with-make-prog --without-make-prog --srcdir=. --curdir --ruby=/home/mastodon/.rbenv/versions/2.5.0/bin/$(RUBY_BASE_NAME) --with-idn-dir --without-idn-dir --with-idn-include --without-idn-include=${idn-dir}/include --with-idn-lib --without-idn-lib=${idn-dir}/lib --with-idnlib --without-idnlib To see why this extension failed to compile, please check the mkmf.log which can be found here: /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/mkmf.log extconf failed, exit code 1 Gem files will remain installed in /home/mastodon/live/vendor/bundle/ruby/2.5.0/gems/idn-ruby-0.1.0 for inspection. Results logged to /home/mastodon/live/vendor/bundle/ruby/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-ruby-0.1.0/gem_make.out An error occurred while installing idn-ruby (0.1.0), and Bundler cannot continue. Make sure that `gem install idn-ruby -v '0.1.0'` succeeds before bundling. In Gemfile: idn-ruby Ohhkayy... well... let's try pointing it directly at the idn library file... bundle install --deployment --without development test --with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11 Response: Unknown switches '--with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11' Okay, so the error message is lying to me, or I'm misreading it. bundle --help gives man-style help which lists all the bundle commands; bundle install --help lists options specific to bundle install, and none of the --with-idn-* options appear to be among either of them. How about: gem install idn-ruby... nope, same error message... How about gem install idn-ruby --with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11... nope: invalid option: --with-idn-lib=/usr/lib/x86_64-linux-gnu/libidn.so.11 part 3 Some research by Ben Hamill suggested this: gem install --remote idn The resulting error message pointed me at a log file, which seemed to be saying that the problem was coming from gcc. Since this means there was compiling involved, I thought maybe I should install the header file for idn as well (as root): apt install libghc-gnuidn-dev. Followed by... su - mastodon gem install --remote idn Output: Building native extensions. This could take a while... ERROR: Error installing idn: ERROR: Failed to build gem native extension. current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext /home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-23716-rrnxuy.rb extconf.rb checking for -lidn... yes checking for idna.h... yes checking for punycode.h... yes checking for stringprep.h... yes creating Makefile current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext make "DESTDIR=" clean current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext make "DESTDIR=" compiling idn.c compiling idna.c idna.c: In function ‘toASCII’: idna.c:88:37: error: ‘struct RString’ has no member named ‘ptr’ rc = idna_to_ascii_8z(RSTRING(str)->ptr, &buf, flags); ^ idna.c: In function ‘toUnicode’: idna.c:128:41: error: ‘struct RString’ has no member named ‘ptr’ rc = idna_to_unicode_8z8z(RSTRING(str)->ptr, &buf, flags); ^ idna.c: At top level: cc1: warning: unrecognized command line option ‘-Wno-self-assign’ cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’ cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’ cc1: warning: unrecognized command line option ‘-Wno-tautological-compare’ Makefile:241: recipe for target 'idna.o' failed make: *** [idna.o] Error 1 make failed, exit code 2 Gem files will remain installed in /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2 for inspection. Results logged to /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-0.0.2/gem_make.out Oops, maybe I was supposed to be in ~/live? Logging in again as root... su - mastodon cd live gem install --remote idn Output: Building native extensions. This could take a while... ERROR: Error installing idn: ERROR: Failed to build gem native extension. current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext /home/mastodon/.rbenv/versions/2.5.0/bin/ruby -r ./siteconf20180409-24783-1trgchl.rb extconf.rb checking for -lidn... yes checking for idna.h... yes checking for punycode.h... yes checking for stringprep.h... yes creating Makefile current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext make "DESTDIR=" clean current directory: /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2/ext make "DESTDIR=" compiling idn.c compiling idna.c idna.c: In function ‘toASCII’: idna.c:88:37: error: ‘struct RString’ has no member named ‘ptr’ rc = idna_to_ascii_8z(RSTRING(str)->ptr, &buf, flags); ^ idna.c: In function ‘toUnicode’: idna.c:128:41: error: ‘struct RString’ has no member named ‘ptr’ rc = idna_to_unicode_8z8z(RSTRING(str)->ptr, &buf, flags); ^ idna.c: At top level: cc1: warning: unrecognized command line option ‘-Wno-self-assign’ cc1: warning: unrecognized command line option ‘-Wno-constant-logical-operand’ cc1: warning: unrecognized command line option ‘-Wno-parentheses-equality’ cc1: warning: unrecognized command line option ‘-Wno-tautological-compare’ Makefile:241: recipe for target 'idna.o' failed make: *** [idna.o] Error 1 make failed, exit code 2 Gem files will remain installed in /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/gems/idn-0.0.2 for inspection. Results logged to /home/mastodon/.rbenv/versions/2.5.0/lib/ruby/gems/2.5.0/extensions/x86_64-linux/2.5.0-static/idn-0.0.2/gem_make.out Looks pretty much the same. All I can think is that this is some kind of version conflict caused by me originally installing the wrong version of Ruby (because that's the version the instructions specified), so let's scrap this build and start over.