Tag Archives: software

Installing CasperJS 1.1.4 on AWS (CentOS)

Installing CasperJS to work with PhantomJSs latest version 2.1.1

This is the current status of my test installation. My perviously hacked version of CaserpJS ( instructions are there: Helping CasperJS 1.1.0-beta3 play nice with PhantomJS 2.0.0 ), however it’s time to rev-up to the non-beta version of the code.

casperjs
CasperJS version 1.1.0-beta3 at /usr/lib/node_modules/casperjs, using phantomjs version 2.1.1

As of today:
Screen Shot 2016-02-08 at 10.42.19 AM

Step 1 — Clone CasperJS from Git

Hopefully you already have Git installed, and you are ready to clone:

git clone git://github.com/n1k0/casperjs.git
Cloning into ‘casperjs’…
remote: Counting objects: 14392, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 14392 (delta 0), reused 0 (delta 0), pack-reused 14385
Receiving objects: 100% (14392/14392), 8.50 MiB | 0 bytes/s, done.
Resolving deltas: 100% (8648/8648), done.
Checking connectivity… done.

Step 2 — Perform Installation

Using hints from the Instructions at CasperJS 1.1.0-DEV documentation, I first located my current casper image, moved it aside, then linked the new one into it’s location.

whereis casperjs
casperjs: /usr/bin/casperjs /usr/local/bin/casperjs /opt/n1k0-casperjs-e3a77d0/bin/casperjs /opt/casperjs/bin/casperjs /opt/casperjs/bin/casperjs.exe

mv /usr/bin/casperjs /usr/bin/casperjs.1.1.0-beta3

ln -sf `pwd`/bin/casperjs /usr/bin/casperjs

Step 3 — Verify Casper

Running casper, I checked to ensure it’s on the latest version:

casperjs
CasperJS version 1.1.0-beta5 at /opt/casperjs, using phantomjs version 2.1.1

This looks like it’s good to go.. now CASPER AWAY!!

Installing PhantomJS 2.1.1 on AWS (CentOS)

phantomjs-logoIt’s a gamble to do this, and according to the build script it’s going to take a long time to complete the compile / install of Phantom 2.1.1.

Note: If you are looking for instructions on building for Ubuntu, the steps are different. I’ve documented that process in this post: Installing PhantomJS 2.1.1 on Ubuntu.

Step 1 — install required dependencies

You may or may not have most of these on your AWS / CentOS system. I found that most of these were required to start the PhantomJS build.Here are the ones that I’ve confirmed I needed:

  • autoconf
  • pkgconfig.x86_64
  • python26-pyudev.noarch
  • python26-twisted.noarch
  • sip.x86_64
  • python27-pyudev.noarch
  • python27-twisted.noarch
  • gcc
  • flex
  • bison
  • xorg-x11-server-Xorg.x86_64
  • xorg-x11-server-devel.x86_64
  • xorg-x11-utils.x86_64
  • xorg-x11-proto-devel.noarch
  • sqlite-tcl.x86_64
  • sqlite-devel.x86_64
  • openssl.x86_64
  • crypto-utils.x86_64
  • openssl-devel.x86_64
  • libfontenc.x86_64
  • libfontenc-devel.x86_64
  • fontconfig.x86_64
  • fontconfig-devel.x86_64
  • libicu-devel.x86_64
  • freetype-devel.x86_64
  • libpng-devel.x86_64
  • libjpeg-turbo-devel.x86_64
  • libXext-devel.x86_64
  • libxcb-devel.x86_64
  • xcb-util.x86_64

Installing the packages went smoothly:

sudo yum install autoconf pkgconfig.x86_64 python26-pyudev.noarch python26-twisted.noarch sip.x86_64 python27-pyudev.noarch python27-twisted.noarch gcc flex bison xorg-x11-server-Xorg.x86_64 xorg-x11-server-devel.x86_64 xorg-x11-utils.x86_64 xorg-x11-proto-devel.noarch sqlite-tcl.x86_64 sqlite-devel.x86_64 openssl.x86_64 crypto-utils.x86_64 openssl-devel.x86_64 libfontenc.x86_64 libfontenc-devel.x86_64 fontconfig.x86_64 fontconfig-devel.x86_64 libicu-devel.x86_64 freetype-devel.x86_64 libpng-devel.x86_64 libjpeg-turbo-devel.x86_64 libXext-devel.x86_64 libxcb-devel.x86_64 xcb-util.x86_64

Step 2 — clone the Git repo to local drive:

git clone git://github.com/ariya/phantomjs.git
Cloning into ‘phantomjs’…
remote: Counting objects: 63695, done.
remote: Compressing objects: 100% (37/37), done.
remote: Total 63695 (delta 16), reused 0 (delta 0), pack-reused 63657
Receiving objects: 100% (63695/63695), 129.05 MiB | 4.08 MiB/s, done.
Resolving deltas: 100% (31013/31013), done.
Checking connectivity… done.

cd phantomjs

git checkout 2.1.1
Note: checking out ‘2.1.1’.
[…]
HEAD is now at d9cda3d… Set version to “2.1.1”

git submodule init
Submodule ‘3rdparty-win’ (https://github.com/Vitallium/phantomjs-3rdparty-win.git) registered for path ‘src/qt/3rdparty’
Submodule ‘qtbase’ (https://github.com/Vitallium/qtbase.git) registered for path ‘src/qt/qtbase’
Submodule ‘qtwebkit’ (https://github.com/Vitallium/qtwebkit.git) registered for path ‘src/qt/qtwebkit’

git submodule update
Cloning into ‘src/qt/3rdparty’…
Cloning into ‘src/qt/qtbase’…
Cloning into ‘src/qt/qtwebkit’…

Step 3 — Hack the QT build

It seemed that I needed to set some different flags for the qtbase build. It was not clear to me if this could be done with the build.py options, so I hacked the qt/qtbase/configure script.

vi src/qt/qtbase/configure

First off, I changed the settings of these two values near the top of the config file:

Then commented out part of the section around Werror, so that the build would not treat warnings as errors. The C++ macro options in the code will generate A LOT of errors, most of them from the flags defined in build.py. I tried the route of disabling those flags and ended up with more errors and more issues.. so changing the flags in the config was my next option:

[…]
#CFG_WERROR=auto
CFG_WERROR=no
[…]
#CFG_DEV=no
CFG_DEV=yes
[…]
warnings-are-errors|Werror)
# if [ “$VAL” = “yes” ] || [ “$VAL” = “no” ]; then
# CFG_WERROR=”$VAL”
# else
UNKNOWN_OPT=yes
# fi
;;
[…]

Step 4 — Build!

python build.py
—————————————-
WARNING
—————————————-

Building PhantomJS from source takes a very long time, anywhere from 30 minutes
to several hours (depending on the machine configuration). It is recommended to
use the premade binary packages on supported operating systems.

For details, please go the the web site: http://phantomjs.org/download.html.

Do you want to continue (Y/n)? Y

Step 5 — check the binary

Once the build has completed, you will find the binary to be built in the local directory bin/

ls -l bin/phantomjs
-rwxr-xr-x 1 root root 56736434 Feb 5 11:33 /usr/sbin/phantomjs

To complete the installation, you’ll need to replace the current phantomjs binary with the new one. To find the location if your current binary (if you have one), this should work:

whereis phantomjs
phantomjs: /usr/bin/phantomjs

Copy the new binary to that location and verify version:

cp bin/phantomjs /usr/bin/phantomjs
cp: overwrite ‘/usr/bin/phantomjs’? y

phantomjs -v
2.1.1

YOU ARE DONE!! It was just that easy

Fun with ‘sed’ taking a list of table names and creating a list of T-SQL TRUNCATE commands

Woo.. I love *NIX, especially ‘sed’ (and awk too for that matter). Wonderful things you can do to automated your programming, speed up development and really REALLY cut down on cut-paste errors.

Today’s task, was to take this list of tables… and generate a single bit fat T-SQL command to truncate them all.. *however* these tables might not exist in a given environment so each Truncate is wrapped in a test to make sure the table is there. If it’s not there, nothing to do, not to mention avoid a potential fatal error trying to truncate a non-existent table.

Here is a snippet of the tables in question:


FCT_CRIMINOGENIC_NEED_TOP_DOMAINS
FCT_DETENTION_RISK_RESPONSE_DETAILS
FCT_DETENTION_RISK_RESPONSE_FULL

This is the desired T-SQL for these three tables

— Truncate table FCT_CRIMINOGENIC_NEED_TOP_DOMAINS
IF EXISTS (SELECT * FROM sys.objects WHERE NAME = ‘FCT_CRIMINOGENIC_NEED_TOP_DOMAINS’)
BEGIN
TRUNCATE TABLE FCT_CRIMINOGENIC_NEED_TOP_DOMAINS
END

— Truncate table FCT_DETENTION_RISK_RESPONSE_DETAILS
IF EXISTS (SELECT * FROM sys.objects WHERE NAME = ‘FCT_DETENTION_RISK_RESPONSE_DETAILS’)
BEGIN
TRUNCATE TABLE FCT_DETENTION_RISK_RESPONSE_DETAILS
END

— Truncate table FCT_DETENTION_RISK_RESPONSE_FULL
IF EXISTS (SELECT * FROM sys.objects WHERE NAME = ‘FCT_DETENTION_RISK_RESPONSE_FULL’)
BEGIN
TRUNCATE TABLE FCT_DETENTION_RISK_RESPONSE_FULL
END

The Sed Command

sed ‘s#^\(.*\)#– Truncate table \1 \’$’\nIF EXISTS (SELECT * FROM sys.objects WHERE NAME = \’\\1\’) \\’$’\n BEGIN \\’$’\n TRUNCATE TABLE \\1 \\’$’\nEND \\’$’\n#g’ MY_INPUT_FILE

Secret Sauce Ingredients

Setting up the string match. In this case it was simple.. I wanted the entire line which is represented with .*

\(.*\)

This loads the matching string into an internal register in ‘sed’ that can be references as ‘1’ within the output replacement expression. This is what it looks like in the command:

\1

Adding a newline in the output.. this was the big trick, and requires dropping into the shell to generate the desired output. The string in the command looks like this:

\’$’\n

You may notice after the first drop into the shell, the escaping has to be doubled for the substitution and the newline…

\\1 \\’$’\n

I cannot speak to the reason for this, but that is what I encountered and why the substitutions change in the latter part of the string. You may find that is not necessary, or might cause an issue in your environment. Adjust as necessary.

Installing CasperJS the compiled way

Installing CaspeJS from source is always fun, so I’m going to tell you how.

Step 1 – Get the source code

Getting the source code from CasperJS website. As of 16-OCT-2015 when I write this, the following command worked. When a new version is released.. this link might go away.. so.. a little hunting might be needed on your part.

The CaseperJS website is located HERE ( http://casperjs.org/ ).

wget https://github.com/n1k0/casperjs/zipball/1.1-beta3/n1k0-casperjs-1.1-beta3-0-g4f105a9.zip

Step 2 – unpack the files

Unpack the files and prepare for compiling:

unzip n1k0-casperjs-1.1-beta3-0-g4f105a9.zip

Step 2 – build it

Installing PhantomJS 2 on Ubuntu

phantomjs-logoIt’s a gamble to do this, and according to the build script it’s going to take a long time to complete the compile / install of Phantom 2.0.

Step 1 — locate the source, download and unzip:

wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.0.0-source.zip
Length: 110092872 (105M) [application/zip]
Saving to: ‘phantomjs-2.0.0-source.zip’

unzip phantomjs-2.0.0-source.zip
Archive: phantomjs-2.0.0-source.zip
a2912c216d06df4d8b51f12ad4082a48c5fc7ba6
creating: phantomjs-2.0.0/
inflating: phantomjs-2.0.0/.gitignore
[…]
inflating: phantomjs-2.0.0/tools/preconfig.sh
inflating: phantomjs-2.0.0/tools/qscriptengine.h
inflating: phantomjs-2.0.0/tools/src.pro

Step 2 — install required dependencies

You may or may not have most of these on your Ubuntu system. I found that most of these were required to start the PhantomJS build.Here are the ones that I’ve confirmed I needed:

  • autoconf2.13
  • ruby
  • pkg-config
  • libicu-dev
  • gperf
  • bison
  • libjpeg-dev
  • g++
  • openssl
  • libtool
  • libssl-dev
  • libpng-dev
  • libpng12-dev
  • libjpeg-dev
  • fontconfig
  • sqlite
  • fontconfig
  • libsqlite0
  • qt5-qmake
  • xorg
  • xorg-dev
  • xutils-dev
  • xcb-proto
  • libxcb-xkb-dev
  • x11proto-core-dev
  • libxcb-render-util0
  • libqt5webkit5-dev

Installing the packages went smoothly:

apt-get install autoconf2.13 ruby pkg-config libicu-dev gperf bison libjpeg-dev g++ openssl libtool libssl-dev libpng-dev libpng12-dev libjpeg-dev fontconfig sqlite fontconfig libsqlite0 qt5-qmake xorg xorg-dev xutils-dev xcb-proto libxcb-xkb-dev x11proto-core-dev libxcb-render-util0 libqt5webkit5-dev

Step 3 — Build Freetype from source and link

Following this I grabbed the source code to install freetype2. Although freetype successfully installed, the required header files where not found. I decided it was bet to grab it and build from source:

wget http://download.savannah.gnu.org/releases/freetype/freetype-2.6.tar.gz
gunzip freetype-2.6.tar.gz
tar xvf freetype-2.6.tar
./configure
[…]
configure: creating ./config.status
config.status: creating unix-cc.mk
config.status: creating unix-def.mk
config.status: creating ftconfig.h
config.status: executing libtool commands
configure:
make & make install
[…]
/usr/bin/install -c -m 644 ./builds/unix/ftconfig.h \
/usr/local/include/freetype2/config/ftconfig.h
/usr/bin/install -c -m 644 /usr/local/freetype-2.6/objs/ftmodule.h \
/usr/local/include/freetype2/config/ftmodule.h
/usr/bin/install -c -m 755 ./builds/unix/freetype-config \
/usr/local/bin/freetype-config
/usr/bin/install -c -m 644 ./builds/unix/freetype2.m4 \
/usr/local/share/aclocal/freetype2.m4
/usr/bin/install -c -m 644 ./builds/unix/freetype2.pc \
/usr/local/lib/pkgconfig/freetype2.pc
/usr/bin/install -c -m 644 /usr/local/freetype-2.6/docs/freetype-config.1 \
/usr/local/share/man/man1/freetype-config.1

Now following that build, due to some inexplicable continuous oversight on the part of freetype’s maintainers.. OR.. phantom.. a link has to be make so that the build process can find the actual libraries required:

ln -s /usr/include/freetype2/freetype /usr/include/freetype

Step 4 — Install an updated version of libxkbcommon

Get the latest version.. and build it:

https://launchpad.net/ubuntu/+archive/primary/+files/libxkbcommon_0.5.0.orig.tar.gz
gunzip libxkbcommon_0.5.0.orig.tar.gz
tar xvf libxkbcommon_0.5.0.orig.tar
cd libxkbcommon-0.5.0
./autogen.sh
./configure

Step 5 — hack QtBase

After many times running thorough the process and resolving errors.. I decided to manually build QTBase. To do this I moved to the source directory for it under PhantomJS, and used this configuration command to get past a libxcb error:

cd src/qt/qtbase
./configure -developer-build -opensource -nomake examples -nomake tests -qt-xcb
./configure -opensource
make
make install

Step 6 — build

Now bulid.sh script. NOTE: if you are executing the compile on a VM (or in this case AWS), it’s recommended that the build process Does Not try to run parallel build jobs on the virtual cores. The PhantomJS website was not clear (to me) why.. but it did recommend using the –jobs 1 flag on the build.. which I am doing. You may omit that if you’d like to experiment.

cd phantomjs-2.0.0

./build.sh –jobs 1
—————————————-
WARNING
—————————————-

Building PhantomJS from source takes a very long time, anywhere from 30
minutes to several hours (depending on the machine configuration).
We recommend you use the premade binary packages on supported operating
systems.

For details, please go the the web site: http://phantomjs.org/download.html.

Do you want to continue (y/n)?
y
[…]

NOTE: If you want to suppress the warning regarding perils of the long compile, you an use the –confirm flag to bypass the question. This is really helpful if you want to background the process and write it to a log. Where I find this most beneficial is when I want to/need to close the terminal window before the compile completes.

Here is an optional method of running that will background the process, auto-reply to the warning and write to a log file:

nohup ./build.sh –confirm –jobs 1 > build.log &

You might carp about not being able to monitor progress now! Well sure you can.. just do a following tail on the file. Exact command varies with system, I’ll provide the one for typical LINUX and for typical OSX:

For typical LINUX:
tailf build.log

For typical OSX:
tail -f build.log

Step 7 — check the binary

Once the build has completed, you will find the binary to be built in the local directory bin/

ls -l bin/phantomjs
-rwxr-xr-x 1 root root 56587060 Sep 30 17:16 bin/phantomjs

To complete the installation, you’ll need to replace the current phantomjs binary with the new one. To find the location if your current binary (if you have one), this should work:

whereis phantomjs
phantomjs: /usr/bin/phantomjs

Copy the new binary to that location and verify version:

cp bin/phantomjs /usr/bin/phantomjs
cp: overwrite ‘/usr/bin/phantomjs’? y

phantomjs -v
2.0.0

YOU ARE DONE!! It was just that easy

PHP’s cURL and how to use CURLPROXY_SOCKS5_HOSTNAME

So, you’re running a proxy (say, something like TOR) on your locallopback.. but you keep getting this error (must have VERBOSE set to true to catch it):

* Hostname was NOT found in DNS cache
* Can’t complete SOCKS5 connection to 0.0.0.0:0. (2)
* Closing connection 0

Your code looks simple enough.. but it doesn’t work:

A little research later you find that the PROPER SOCKS5 protocol is SOCKS5H (hostname), so you use the proper flag..

[…]
curl_setopt($ch, CURLOPT_PROXYTYPE, CURLPROXY_SOCKS5_HOSTNAME);
[…]

only to get told that TOR doesn’t want to be used as an HTTP proxy (what’s the deal)?


It appears you have configured your web browser to use Tor as an HTTP proxy.
This is not correct: Tor is a SOCKS proxy, not an HTTP proxy.
Please configure your client accordingly.

Have not fear script-kiddies.. even though PHP as of 5.5.x does not seem to include this CRITICAL cURL FLAG, you can reference it directly with the integer value of 7. So… if you change your code to this… you’re as golden as the proverbial goose!

curl_setopt($ch, CURLOPT_PROXYTYPE, 7); // 7 = CURLPROXY_SOCKS5_HOSTNAME

So… now you know!

find my article helpful? print some coupons!

Helping CasperJS 1.1.0-beta3 play nice with PhantomJS 2.0.0

As of today, 30-SEP-2015, the latest build for CasperJS 1.1.0-beta3. Not exactly comforting for production use, but it is the most recent and decently capable version available outside of a pull request. So.. let’s get started:

First, I checked the current version of

[root@ip-10-153-205-78 ~]# casperjs –version
1.1.0-beta3
Unsafe JavaScript attempt to access frame with URL about:blank from frame with URL file:///usr/local/lib/node_modules/casperjs/bin/bootstrap.js. Domains, protocols and ports must match.

NOTE:You may notice that simply running casper causes PhantomJS to hurl out worthless warning messages. This is the very reason I’m undergoing this exercise.

Next, I dropped the new binary on target server and verified that I am dealing with pJS 2.0, and it’s functioning:

[root@ip-10-153-205-78 ~]# phantomjs -v
2.0.0

Next step is to get to the meat of the errors.

The first I encountered was this:

[root@ip-10-153-205-78 ~]# casperjs –version
CasperJS needs PhantomJS v1.x

/usr/local/lib/node_modules/casperjs/bin/bootstrap.js:91 in __die

Opening this file, around line 91 the following is found:

function __die(message) {
if (message) {
console.error(message);
}
phantom.exit(1);

Tracing back to the caller, the test is performed here:

(function(version) {
// required version check
if (version.major !== 1) {
return __die(‘CasperJS needs PhantomJS v1.x’);
} if (version.minor < 8) { return __die('CasperJS needs at least PhantomJS v1.8 or later.'); } if (version.minor === 8 && version.patch < 1) { return __die('CasperJS needs at least PhantomJS v1.8.1 or later.'); } })(phantom.version);

Next I tried to make it accept version 2, by changing that block to this:

(function(version) {
if (version.major == 1) {
if (version.minor < 8) { return __die('CasperJS needs at least PhantomJS v1.8 or later.'); } if (version.minor === 8 && version.patch < 1) { return __die('CasperJS needs at least PhantomJS v1.8.1 or later.'); } } if (version.major < 2) { return __die('CasperJS needs PhantomJS v1.x or v2.x'); } })(phantom.version);

Next error message was this:

[root@ip-10-153-205-78 ~]# casperjs –version
Couldn’t find nor compute phantom.casperPath, exiting.

/usr/local/lib/node_modules/casperjs/bin/bootstrap.js:91 in __die

It’s origin was in this block:

// CasperJS root path
if (!phantom.casperPath) {
try {
phantom.casperPath = phantom.args.map(function _map(arg) {
var match = arg.match(/^–casper-path=(.*)/);
if (match) {
return fs.absolute(match[1]);
}
}).filter(function _filter(path) {
return fs.isDirectory(path);
}).pop();
} catch (e) {
return __die(“Couldn’t find nor compute phantom.casperPath, exiting.”);
}
}

Based upon information found in this post.. Latest pull of casperjs not working with latest pull of phantomjs2 I made the following modifications:

Paste this section of code in above the first non-comment section in bootstrap file:

// Mods to get Casper and PhantomJS playing nice
var system = require(‘system’);
var argsdeprecated = system.args;
argsdeprecated.shift();
phantom.args = argsdeprecated;

Once you have done that, you should be able to use Casper 1.1.0-beta3 with PhantomJS 2.0 (and look.. NO MORE LAME WARNINGS!!!)

[root@ip-10-153-205-78 ~]# casperjs –version
1.1.0-beta3

Installing Gearman on OSX Yosemite (usually)

logo

Another round of Gearman installs, following the update to OSX Yosemite. (Updated 12-AUG-2015)

Here is your guide to getting it done!

Make sure you have XCode developer tools

First step is to make sure you have the Command Line developer tools installed. To do this, or verify that it’s already done, while logged in as a non-root user, type:

xcode-select –install

A system dialog box should open up and request that you grant it permission to perform the command line install. Follow the steps and instructions in the dialogs to complete this step.

Collected the Packages

You will need the following packages

  • libevent
  • boost
  • gearmand

libevent

The latest version can be acquired here: http://libevent.org/

Unpack and compile

gunzip libevent-2.0.22-stable.tar.gz

tar xvf libevent-2.0.22-stable.tar

./configure

make install

boost

The latest version can be acquired here:
http://www.boost.org/users/download/

Unpack and compile

gunzip boost_1_58_0.tar.gz

tar xvf boost_1_58_0.tar

./bootstrap.sh

./b2 –a –build-type=complete –layout=versioned

Note: these paths reported by b2 during build are important to save:

The Boost C++ Libraries were successfully built!

The following directory should be added to compiler include paths:

/opt/boost_1_58_0

The following directory should be added to linker library paths:

/opt/boost_1_58_0/stage/lib

Gearman

The latest code can be acquired here: https://launchpad.net/gearmand

It is possible that the compiler won’t find libevent so the following environment variables may need to be set:

export CPPFLAGS=’-I/opt/boost_1_58_0′
export LDFLAGS=’-L/opt/boost_1_58_0/stage/lib’

Unpack and compile

gunzip gearmand-1.1.12.tar.tgz

tar xvf gearmand-1.1.12.tar

./configure –with-boost –with-boost-libdir=/opt/boost_1_58_0/stage/lib –prefix=/opt –with-sanitize –enable-fast-install –with-gnu-ld –enable-ssl

make && make install

At this point, you should be up and running!

If you also need to install gearman for PHP libraries, my post here should help: https://blog.daviddemartini.com/archives/5312

Install Redis on AWS EC2

redis-whiteRedis is fairly simple to install and get running. I found the best way to do this on CentOS based AWS EC2 nodes is to use the following steps.

Install Pre-Requisites

Redis will require several per-requisits. Your system may vary, but these are the cases I ran into when running the build in August 2015 with the latest AWS system updates. Some of these are required to run the tests, others are required for Redis itself.

TCL 8.5 or higher for Test

You need tcl 8.5 or newer in order to run the Redis test

yum install tcl

Download latest Redis package

Assume super user, move to a safe directory (I like /usr/local) and download the latest build:

sudo su –
cd /usr/local
wget http://download.redis.io/redis-stable.tar.gz

Extract Files

Once the main tarball has been downloaded, extract the files and start the configuration process.

tar xvzf redis-stable.tar.gz
cd redis-stable

Build the Binary

Build the binary. Redis does not seem to require ./config to be run, the necessary make files are already in place. Just run make and install!! If you decide to run the ‘make test’ (which I suggest you do), it maybe take 10-15 min. to complete depending on the power of your AWS instance.

make
make test
make install

Set Overcommit to TRUE

Redis is going to complain unless you have some level of overcommit memory enabled. This is easy to do (again, you must be root or sudoer to do this). Add ‘vm.overcommit_memory = 1’ to /etc/sysctl.conf and then reboot, IF you can safely do so on your machine (best to check and make sure there are no live service interruptions or other personnel using the system).

vi /etc/sysctl.conf

Add this to the end of the file:

# Required by Redis to enable overcommit setting:
vm.overcommit_memory = 1

Reboot

init 6

Configure Redis

Create a working directory for the redis disk files. I like to use the following:

mkdir /var/redis
mkdir /var/redis/db

Copy the base configuration file to /etc/ and customize to your environment.

mkdir /etc/redis
cp redis.conf /etc/redis/6379.conf
vi /etc/redis/6379.conf

I made the following changes to the configuration file. I can’t guarantee all or any of these will be correct for your configuration:

daemonize yes

bind 127.0.0.1

tcp-keepalive 60

logfile “/var/log/redis-server.log”

dir /var/redis/db

Copy the startup file into /etc/init.d

cp utils/redis_init_script /etc/init.d/redis

Add the start command to the root’s crontab. Yeah, so this might be a cheater method instead of adding this to the systems rd.X files, but it’s also easy to disable.

crontab -e

@reboot /etc/init.d/redis start

Start Redis Server

Starting the server from the command line is a good way to verify it’s functional. It’s easy to do, just type ‘resis-server’. Hit CNTL-C to kill and exit once you’ve tested launch. If it starts up, you should see something like this:

[root@ip-10-000-000-00 redis-stable]# redis-server

31408:C 04 Aug 21:55:00.578 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
31408:M 04 Aug 21:55:00.579 * Increased maximum number of open files to 10032 (it was originally set to 1024).

31408:M 04 Aug 21:55:00.581 # Server started, Redis version 3.0.3

31408:signal-handler (1438725473) Received SIGINT scheduling shutdown…
31408:M 04 Aug 21:57:53.628 # User requested shutdown…
31408:M 04 Aug 21:57:53.628 * Saving the final RDB snapshot before exiting.
31408:M 04 Aug 21:57:53.631 * DB saved on disk
31408:M 04 Aug 21:57:53.632 # Redis is now ready to exit, bye bye…

If that looks OK, then start using the startup back file. This should start redis as a deamon (service) depending on how you edited the configuration file. If you did it the way I did, then it will start as a deamon.

/etc/init.d/redis start

Starting Redis server…

Test to make sure it’s listening.. by using the ping command. If it’s alive and listening, you’ll receive back a ‘PONG’

redis-cli ping

PONG

FINAL STEPS — Reboot and Verify!

A good and proper final test, assuming you are able to reboot the system without causing trouble to any live services or other personnel… is.. REBOOT, then verify that it has restarted as expected.

init 6

Connection closed by remote host.

[ec2-user@ip-10-000-000-00 ~]$ redis-cli ping

PONG

CONGRATULATIONS!! You are now the proud owner/maintainer/RP of a Redis server!

NEXT…

Doing something productive with Redis… (to be continued)