Helping CasperJS 1.1.0-beta3 play nice with PhantomJS 2.0.0

As of today, 30-SEP-2015, the latest build for CasperJS 1.1.0-beta3. Not exactly comforting for production use, but it is the most recent and decently capable version available outside of a pull request. So.. let’s get started:

First, I checked the current version of

[root@ip-10-153-205-78 ~]# casperjs –version
1.1.0-beta3
Unsafe JavaScript attempt to access frame with URL about:blank from frame with URL file:///usr/local/lib/node_modules/casperjs/bin/bootstrap.js. Domains, protocols and ports must match.

NOTE:You may notice that simply running casper causes PhantomJS to hurl out worthless warning messages. This is the very reason I’m undergoing this exercise.

Next, I dropped the new binary on target server and verified that I am dealing with pJS 2.0, and it’s functioning:

[root@ip-10-153-205-78 ~]# phantomjs -v
2.0.0

Next step is to get to the meat of the errors.

The first I encountered was this:

[root@ip-10-153-205-78 ~]# casperjs –version
CasperJS needs PhantomJS v1.x

/usr/local/lib/node_modules/casperjs/bin/bootstrap.js:91 in __die

Opening this file, around line 91 the following is found:

function __die(message) {
if (message) {
console.error(message);
}
phantom.exit(1);

Tracing back to the caller, the test is performed here:

(function(version) {
// required version check
if (version.major !== 1) {
return __die(‘CasperJS needs PhantomJS v1.x’);
} if (version.minor < 8) { return __die('CasperJS needs at least PhantomJS v1.8 or later.'); } if (version.minor === 8 && version.patch < 1) { return __die('CasperJS needs at least PhantomJS v1.8.1 or later.'); } })(phantom.version);

Next I tried to make it accept version 2, by changing that block to this:

(function(version) {
if (version.major == 1) {
if (version.minor < 8) { return __die('CasperJS needs at least PhantomJS v1.8 or later.'); } if (version.minor === 8 && version.patch < 1) { return __die('CasperJS needs at least PhantomJS v1.8.1 or later.'); } } if (version.major < 2) { return __die('CasperJS needs PhantomJS v1.x or v2.x'); } })(phantom.version);

Next error message was this:

[root@ip-10-153-205-78 ~]# casperjs –version
Couldn’t find nor compute phantom.casperPath, exiting.

/usr/local/lib/node_modules/casperjs/bin/bootstrap.js:91 in __die

It’s origin was in this block:

// CasperJS root path
if (!phantom.casperPath) {
try {
phantom.casperPath = phantom.args.map(function _map(arg) {
var match = arg.match(/^–casper-path=(.*)/);
if (match) {
return fs.absolute(match[1]);
}
}).filter(function _filter(path) {
return fs.isDirectory(path);
}).pop();
} catch (e) {
return __die(“Couldn’t find nor compute phantom.casperPath, exiting.”);
}
}

Based upon information found in this post.. Latest pull of casperjs not working with latest pull of phantomjs2 I made the following modifications:

Paste this section of code in above the first non-comment section in bootstrap file:

// Mods to get Casper and PhantomJS playing nice
var system = require(‘system’);
var argsdeprecated = system.args;
argsdeprecated.shift();
phantom.args = argsdeprecated;

Once you have done that, you should be able to use Casper 1.1.0-beta3 with PhantomJS 2.0 (and look.. NO MORE LAME WARNINGS!!!)

[root@ip-10-153-205-78 ~]# casperjs –version
1.1.0-beta3

Upgrade PhantomJS 1.9 to 2.0 on AWS

phantomjs-logoIt’s a gamble to do this, and according to the build script it’s going to take a long time to complete, but to try and solve some issues that PhantomJS has with CasperJS 1.1-beta3 (latest version) I wanted to upgrade to Phantom 2.0.

A lot of things have changed, and it’s been suggested that a number of features that CasperJS wants to use, are deprecated in the 2.0 version of Phantom. But forward I’ll forge regardless.

Step 1 is to locate the source, download and unzip:

wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.0.0-source.zip
Length: 110092872 (105M) [application/zip]
Saving to: ‘phantomjs-2.0.0-source.zip’

unzip phantomjs-2.0.0-source.zip
Archive: phantomjs-2.0.0-source.zip
a2912c216d06df4d8b51f12ad4082a48c5fc7ba6
creating: phantomjs-2.0.0/
inflating: phantomjs-2.0.0/.gitignore
[…]
inflating: phantomjs-2.0.0/tools/preconfig.sh
inflating: phantomjs-2.0.0/tools/qscriptengine.h
inflating: phantomjs-2.0.0/tools/src.pro

Step 2 – install required dependancies

You may or may not have most of these from your previous PhantomJS 1.9.x install, but I found that most of these were required to start the PhantomJS build.Here are the ones that I’ve confirmed I needed:

  • gcc
  • gcc-c++
  • make
  • flex
  • ruby
  • openssl-devel
  • fontconfig-devel
  • sqlite-devel
  • libicu-devel
  • libpng-devel
  • libjpeg-devel
  • freetype-devel
  • bison
  • gperf

Installing the packages went smoothly:

sudo yum -y install gcc gcc-c++ make flex bison gperf ruby openssl-devel freetype-devel fontconfig-devel libicu-devel sqlite-devel libpng-devel libjpeg-devel

Following this I grabbed the source code to install freetype2. Although freetype successfully installed, the required header files where not found. I decided it was bet to grab it and build from source:

wget http://download.savannah.gnu.org/releases/freetype/freetype-2.6.tar.gz
gunzip freetype-2.6.tar.gz
tar xvf freetype-2.6.tar
./configure
[…]
configure: creating ./config.status
config.status: creating unix-cc.mk
config.status: creating unix-def.mk
config.status: creating ftconfig.h
config.status: executing libtool commands
configure:
make & make install
[…]
/usr/bin/install -c -m 644 ./builds/unix/ftconfig.h \
/usr/local/include/freetype2/config/ftconfig.h
/usr/bin/install -c -m 644 /usr/local/freetype-2.6/objs/ftmodule.h \
/usr/local/include/freetype2/config/ftmodule.h
/usr/bin/install -c -m 755 ./builds/unix/freetype-config \
/usr/local/bin/freetype-config
/usr/bin/install -c -m 644 ./builds/unix/freetype2.m4 \
/usr/local/share/aclocal/freetype2.m4
/usr/bin/install -c -m 644 ./builds/unix/freetype2.pc \
/usr/local/lib/pkgconfig/freetype2.pc
/usr/bin/install -c -m 644 /usr/local/freetype-2.6/docs/freetype-config.1 \
/usr/local/share/man/man1/freetype-config.1

Now following that build, due to some inexplicable continuous oversight on the part of freetype’s maintainers.. OR.. phantom.. a link has to be make so that the build process can find the actual libraries required:

ln -s /usr/include/freetype2/freetype /usr/include/freetype

Step 3 – build

Now bulid.sh script. NOTE: if you are executing the compile on a VM (or in this case AWS), it’s recommended that the build process Does Not try to run parallel build jobs on the virtual cores. The PhantomJS website was not clear (to me) why.. but it did recommend using the –jobs 1 flag on the build.. which I am doing. You may omit that if you’d like to experiment.

cd phantomjs-2.0.0

./build.sh –jobs 1
—————————————-
WARNING
—————————————-

Building PhantomJS from source takes a very long time, anywhere from 30
minutes to several hours (depending on the machine configuration).
We recommend you use the premade binary packages on supported operating
systems.

For details, please go the the web site: http://phantomjs.org/download.html.

Do you want to continue (y/n)?
y
[…]

NOTE: If you want to suppress the warning regarding perils of the long compile, you an use the –confirm flag to bypass the question. This is really helpful if you want to background the process and write it to a log. Where I find this most beneficial is when I want to/need to close the terminal window before the compile completes.

Here is an optional method of running that will background the process, auto-reply to the warning and write to a log file:

nohup ./build.sh –confirm –jobs 1 > build.log &

You might carp about not being able to monitor progress now! Well sure you can.. just do a following tail on the file. Exact command varies with system, I’ll provide the one for typical LINUX and for typical OSX:

For typical LINUX:
tailf build.log

For typical OSX:
tail -f build.log

Step 4 – check the binary

Once the build has completed, you will find the binary to be built in the local directory bin/

ls -l bin/phantomjs
-rwxr-xr-x 1 root root 56587060 Sep 30 17:16 bin/phantomjs

To complete the installation, you’ll need to replace the current phantomjs binary with the new one. To find the location if your current binary (if you have one), this should work:

whereis phantomjs
phantomjs: /usr/bin/phantomjs

Copy the new binary to that location and verify version:

cp bin/phantomjs /usr/bin/phantomjs
cp: overwrite ‘/usr/bin/phantomjs’? y

phantomjs -v
2.0.0

YOU ARE DONE!! It was just that easy

F-22 Raptor at the California International Air Show

Below are photographs of the Air Force’s latest superiority fighter, the F-22 Raptor. In addition, there are also photos of the USAF Heritage Fly-by with the F-22 and P-51 Mustang in tight formation.

Photos taken at the 2015 California International Air Show – Salinas California.

F-22 Raptor


F-22  Raptor
F-22 Raptor showing some vapor during a high speed turn.

F-22  Raptor
F-22 Raptor right at the threshold of MACH 1.
F-22  Raptor
F-22 Raptor with afterburners lit
F-22  Raptor
F-22 Raptor in a high-speed pass.

USAF Heritage Fly-by (F-22 & P-51)


F-22 & P-51 Heritage Fly-by
F-22 & P-51 Heritage Fly-by

F-22 & P-51 Heritage Fly-by
F-22 & P-51 Heritage Fly-by

SOLR 5 — Fixing SSL WEAK SERVER EPHEMERAL DH_KEY

I ran into this short stopper today, which prevented access to the Admin interfaces of my Solr indexer:
ERR_SSL_WEAK_SERVER_EPHEMERAL_DH_KEY

Screen Shot 2015-09-09 at 10.48.15 AM

Recalling how difficult it was to first enable SSL with my Solr cluster.. I suspected this would be a major issue (and for the most part it was)

STACK OVERFLOW TO THE RESCUE!
Using this article: How to fix ERR_SSL_WEAK_SERVER_EPHEMERAL_DH_KEY I was able to modify my Solr 5 / Jetty configuration to stop use of the weaker DH keys.

The first solution using the wildcarded names did not work for me. After a couple of hours of testing and looking at my logs, I found that another un-voted solution fixed the issue without much hassle.

The Fix

The configuration file I modified to resolve this is the jetty-https-ssl.xml. It is located in /opt/solr/server/etc on my server.. your situation might be a little different (you’ll need to find that file that is in use by Solr yourself if it’s not in the above location).

[root@]cd /opt/solr/server/etc
[root@]vi jetty-https-ssl.xml

Locate the block where you have your SSL config. This is what mine looks like:

<Call name=”addConnector”>
<Arg>
<New class=”org.eclipse.jetty.server.ssl.SslSelectChannelConnector”>
<Arg>
<New class=”org.eclipse.jetty.http.ssl.SslContextFactory”>

<Set name=”keyStore”><SystemProperty name=”jetty.home” default=”.”/>/etc/solr-ssl.keystore.jks</Set>
<Set name=”keyStorePassword”>9290j2039fh09209h390h8f23</Set>
<Set name=”needClientAuth”><SystemProperty name=”jetty.ssl.clientAuth” default=”false”/></Set>

</New>
</Arg>

<Set name=”port”><SystemProperty name=”jetty.ssl.port” default=”8984″/></Set>
<Set name=”maxIdleTime”>30000</Set>
</New>
</Arg>
</Call>

 

Within the NEW block, below existing values.. I added this complete section, verbatim:


<!–  //START ADDED   9-SEP-2015 for  Diffie-Hellman  group size Fix –>

<Set name=”ExcludeCipherSuites”>
<Array type=”String”>
<Item>SSL_RSA_WITH_DES_CBC_SHA</Item>
<Item>SSL_DHE_RSA_WITH_DES_CBC_SHA</Item>
<Item>SSL_DHE_DSS_WITH_DES_CBC_SHA</Item>
<Item>SSL_RSA_EXPORT_WITH_RC4_40_MD5</Item>
<Item>SSL_RSA_EXPORT_WITH_DES40_CBC_SHA</Item>
<Item>SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA</Item>
<!– Disable small Diffie-Hellman key exchange to prevent Logjam attack —>
<Item>SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA</Item>
<Item>SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA</Item>
<Item>TLS_DHE_RSA_WITH_AES_256_CBC_SHA256</Item>
<Item>TLS_DHE_DSS_WITH_AES_256_CBC_SHA256</Item>
<Item>TLS_DHE_RSA_WITH_AES_256_CBC_SHA</Item>
<Item>TLS_DHE_DSS_WITH_AES_256_CBC_SHA</Item>
<Item>TLS_DHE_RSA_WITH_AES_128_CBC_SHA256</Item>
<Item>TLS_DHE_DSS_WITH_AES_128_CBC_SHA256</Item>
<Item>TLS_DHE_RSA_WITH_AES_128_CBC_SHA</Item>
<Item>TLS_DHE_DSS_WITH_AES_128_CBC_SHA</Item>
</Array>
</Set>

<Set name=”ExcludeProtocols”>
<Array type=”java.lang.String”>
<Item>SSLv3</Item>
</Array>
</Set>

<!–  //END  Diffie-Hellman  group size Fix                          –>

Once that was added, I saved the file and I restarted solr:

[root@]cd /opt/solr/server/etc

Sending stop command to Solr running on port 8984 … waiting 5 seconds to allow Jetty process 3861 to stop gracefully.
Waiting to see Solr listening on port 8984 [-]
Started Solr server on port 8984 (pid=4162). Happy searching!

Re-trying my Solr Admin… BACK IN BUSINESS!!! 😀
Screen Shot 2015-09-09 at 11.02.26 AM