1042183 - Figure out best way to handle libzmq dependency from on-device gaia-integration tests

Status: RESOLVED FIXED

(Firefox OS Graveyard :: Gaia::Build, defect)

Description

[Andrew Halberstadt [:ahal]]

Bug 994764 tracks using a socket to connect to our python automation toolchain so we can more easily run gaia-integration tests on devices and emulators. It uses ZeroMQ which means it introduces a dependency on libzmq.

There are several ways we can approach this dependency:

1) Add it to the build prerequisites (it is available in most if not all package repositories). If a user neglects to install this and then tries to run something that requires it, we would detect that it is missing and provide a clear error with steps on how to install it.

2) Install it automatically if we don't detect it. Probably the easiest way to do this cross-platform is to download the official tarball and compile it from source. I already have a patch that does this, which I'll upload shortly. The downside to this approach is that we need to install it in user space which may cause complications.

3) Hybrid approach. Another alternative is to still provide the make target to install it, but not invoke it automatically. This requires some attention from the user, but means they can run the command as root. The benefit of this over the first approach is that we can control which version of the library gets used (as opposed to whatever version happens to be in the user's package repos).

Another thing to think about, is what commands "depend" on libzmq. Technically only gaia-integration tests do, but the NodeJS zmq bindings throw an error if you try to install them without the core library. This means we can't put it (or anything that depends on it) in gaia-node-modules without causing bustage.

Finally there's the question of test slaves. Ideally we'd just update their image to have the library pre-installed. Not sure how difficult that would be.

Comment 1

[Andrew Halberstadt [:ahal]]

Attachment: Make target to download libzmq and install from source

This patch adds a make target to install libzmq and makes the node_modules target depend on it. Excuse my shoddy scripting :p

I don't really know if doing this is a good idea or not yet.

Comment 2

[Andrew Halberstadt [:ahal]]

Actually, this works for the NodeJS bindings, but pyzmq doesn't seem to be able to find it (I guess it doesn't use pkg-config?). Unless I can figure out why, option 2 might be a non-starter.

Comment 3

[Jonathan Griffin (:jgriffin)]

For on-device tests, I think we can file a bug to have libzmq added.  For AWS buildbot nodes, we can probably get RelEng to add this package as well, via a separate bug.

Comment 4

[Andrew Halberstadt [:ahal]]

(In reply to Jonathan Griffin (:jgriffin) from comment #3)
> For on-device tests, I think we can file a bug to have libzmq added.  For
> AWS buildbot nodes, we can probably get RelEng to add this package as well,
> via a separate bug.

You mean added to the jenkins slaves?

One thing to keep in mind is that we can't land bug 1037089 until this is solved everywhere that is even invoking "make node_modules". This means Jenkins, AWS, local dev machines, etc..

An alternative is to install marionette-socket-host, mocha-socket-reporter and corredor-js separately from gaia-node-modules. This way we could set it up so only things that want to run Gi on devices need to have libzmq.

Comment 5

[Jonathan Griffin (:jgriffin)]

Yes, I was thinking of the Jenkins slaves.  We have a pretty good degree of flexibility with what we can do there; buildbot slave change are more difficult (but not too bad for AWS nodes), and of course the local case may be hardest of all.

For the local case, I might think that erroring out if the libzmq isn't installed is a viable option, if we can't make #2 work.

Comment 6

[Jonathan Griffin (:jgriffin)]

(In reply to Andrew Halberstadt [:ahal] from comment #4)

> An alternative is to install marionette-socket-host, mocha-socket-reporter
> and corredor-js separately from gaia-node-modules. This way we could set it
> up so only things that want to run Gi on devices need to have libzmq.

Let's set up the Jenkins slaves this way for now; that will allow us to get the tests running there soonest, I think, so we can see how well they run and how much other work is left to do.

Comment 7

[Andrew Halberstadt [:ahal]]

(In reply to Jonathan Griffin (:jgriffin) from comment #6)
> Let's set up the Jenkins slaves this way for now; that will allow us to get
> the tests running there soonest, I think, so we can see how well they run
> and how much other work is left to do.

I think that makes the most sense. In that case there is nothing here really blocking us, just a matter of adding a step to install the library if it doesn't exist. I've been using "pkg-config libzmq --exists" to test it (a return value of 0 means it is installed).

> For the local case, I might think that erroring out if the libzmq isn't
> installed is a viable option, if we can't make #2 work.

Yes, I think option #3 is what I'm leaning towards the most.

Comment 8

[Stephen Donner [:stephend] Not actively reading bugmail]

(In reply to Andrew Halberstadt [:ahal] from comment #7)
> (In reply to Jonathan Griffin (:jgriffin) from comment #6)
> > Let's set up the Jenkins slaves this way for now; that will allow us to get
> > the tests running there soonest, I think, so we can see how well they run
> > and how much other work is left to do.
> 
> I think that makes the most sense. In that case there is nothing here really
> blocking us, just a matter of adding a step to install the library if it
> doesn't exist. I've been using "pkg-config libzmq --exists" to test it (a
> return value of 0 means it is installed).

Dylan's installed this on a few nodes, but I've pinged again in bug 1042950 to have it deployed across a couple more nodes, at least, to get us unblocked on this piece.

Comment 9

[Malini Das [:mdas] - Away, not checking bugmail]

Attachment: link to pr

Thanks for the patch ahal, I made this patch off of it.

This patch tackles both Bug 1042183 and Bug 1037089 by installing zmq (needed for device integration tests), installing the needed npm packages for device tests, and turning on the virtualenv (device tests use a python service to manage the device).

Tested it on Linux and OSX.

Comment 10

[Ricky Chien [:rickychien] (inactive)]

Comment on attachment 8499817 [details] [review]
link to pr

For bin/gaia-marionette, we should request Evan to review this part.

Comment 11

[Ricky Chien [:rickychien] (inactive)]

Comment on attachment 8499817 [details] [review]
link to pr

I've leaved comments on Github and it seems to be no problem after fixing nits.

r=@RickyChien thanks!

Comment 12

[Malini Das [:mdas] - Away, not checking bugmail]

Comment on attachment 8499817 [details] [review]
link to pr

Thanks for the r+ Ricky, but I just noticed a bug when trying to run this locally after uninstalling zmq. I'm going to fix this and then r? after.

Comment 13

[Malini Das [:mdas] - Away, not checking bugmail]

fixed as a byproduct of Bug 1088835.

We now install libzmq and all that fun stuff as a preinstall step for marionette_socket_host.