Skip to content

Commit 31072ce

Browse files
mdmintzmdmintz
committed
Merge pull request #7 from mdmintz/docker_overhaul
Make headless browser automation easier
2 parents b382d58 + f62dcea commit 31072ce

15 files changed

+127
-192
lines changed

‎Docker_README.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -6,41 +6,41 @@
66

77
docker-machine create --driver virtualbox seleniumbase
88

9-
#### 3. If your Docker environment ever goes down for any reason, you can bring it back up with a restart:
9+
##### (If your Docker environment ever goes down for any reason, you can bring it back up with a restart.)
1010

1111
docker-machine restart seleniumbase
1212

13-
#### 4. Configure your shell:
13+
#### 3. Configure your shell:
1414

1515
eval "$(docker-machine env seleniumbase)"
1616

17-
#### 5. Go to the SeleniumBase home directory. (That's where "Dockerfile" is located)
17+
#### 4. Go to the SeleniumBase home directory. (That's where "Dockerfile" is located)
1818

19-
#### 6. Create your Docker image from your Dockerfile: (Get ready to wait awhile)
19+
#### 5. Create your Docker image from your Dockerfile: (Get ready to wait awhile)
2020

2121
docker build -t seleniumbase .
2222

23-
#### 7. Run a test inside your Docker: (Once the test completes after a few seconds, you'll automatically exit the Docker shell)
23+
#### 6. Run a test inside your Docker: (Once the test completes after a few seconds, you'll automatically exit the Docker shell)
2424

2525
docker run seleniumbase ./run_docker_test_in_firefox.sh
2626

27-
#### 8. Now run the same test with Chrome inside your Docker:
27+
#### 7. Now run the same test with Chrome inside your Docker:
2828

2929
docker run seleniumbase ./run_docker_test_in_chrome.sh
3030

31-
#### 9. You can also enter Docker and stay inside the shell:
31+
#### 8. You can also enter Docker and stay inside the shell:
3232

3333
docker run -i -t seleniumbase
3434

35-
#### 10. Now you can run the example test from inside the Docker shell: (This time with PhantomJS)
35+
#### 9. Now you can run the example test from inside the Docker shell:
3636

37-
./run_docker_test_in_phantomjs.sh
37+
./run_docker_test_in_chrome.sh
3838

39-
#### 11. When you're satisfied, you may exit the Docker shell:
39+
#### 10. When you're satisfied, you may exit the Docker shell:
4040

4141
exit
4242

43-
#### 12. (Optional) Since Docker images and containers take up a lot of space, you may want to clean up your machine from time to time when they’re not being used:
43+
#### 11. (Optional) Since Docker images and containers take up a lot of space, you may want to clean up your machine from time to time when they’re not being used:
4444
http://stackoverflow.com/questions/17236796/how-to-remove-old-docker-containers
4545
Here are a few of those cleanup commands:
4646

@@ -57,7 +57,7 @@ Finally, if you want to wipe out your SeleniumBase Docker virtualbox, use these
5757
docker-machine kill seleniumbase
5858
docker-machine rm seleniumbase
5959

60-
#### 13. (Optional) More reading on Docker can be found here:
60+
#### 12. (Optional) More reading on Docker can be found here:
6161
* https://docs.docker.com
6262
* https://docs.docker.com/mac/started/
6363
* https://docs.docker.com/installation/mac/

‎Dockerfile

Lines changed: 54 additions & 57 deletions
Original file line numberDiff line numberDiff line change
@@ -1,40 +1,29 @@
11
# SeleniumBase Docker Image
22
FROM ubuntu:14.04
33

4+
#=======================================
45
# Install Python and Basic Python Tools
6+
#=======================================
57
RUN apt-get update && apt-get install -y python python-pip python-setuptools python-dev python-distribute
68

7-
#========================
8-
# Miscellaneous packages
9-
# Includes minimal runtime used for executing selenium with firefox
10-
#========================
11-
ENV BUILD_DEPS '\
12-
build-essential \
13-
libmysqlclient-dev \
14-
libpython-dev \
15-
libyaml-dev \
16-
libxml2-dev \
17-
libxslt1-dev \
18-
libxslt-dev \
19-
zlib1g-dev \
20-
'
21-
22-
RUN apt-get update -qqy \
23-
&& apt-get -qy --no-install-recommends install \
24-
locales \
25-
language-pack-en \
9+
#=================================
10+
# Install Bash Command Line Tools
11+
#=================================
12+
RUN apt-get -qy --no-install-recommends install \
2613
sudo \
2714
unzip \
2815
wget \
2916
curl \
3017
vim \
3118
xvfb \
32-
libaio1 \
33-
libxml2 \
34-
libxslt1.1 \
35-
mysql-client \
36-
${BUILD_DEPS} \
37-
&& rm -rf /var/lib/apt/lists/*
19+
&& rm -rf /var/lib/apt/lists/*
20+
21+
#========================================
22+
# Add normal user with passwordless sudo
23+
#========================================
24+
RUN sudo useradd seluser --shell /bin/bash --create-home \
25+
&& sudo usermod -a -G sudo seluser \
26+
&& echo 'ALL ALL = (ALL) NOPASSWD: ALL' >> /etc/sudoers
3827

3928
#==============================
4029
# Locale and encoding settings
@@ -44,19 +33,16 @@ ENV LANG ${LANGUAGE}
4433
RUN locale-gen ${LANGUAGE} \
4534
&& dpkg-reconfigure --frontend noninteractive locales
4635

47-
#====================
48-
# Firefox Latest ESR
49-
#====================
50-
RUN apt-get update -qqy \
51-
&& apt-get -qy --no-install-recommends install \
52-
$(apt-cache depends firefox | grep Depends | sed "s/.*ends:\ //" | tr '\n' ' ') \
53-
&& rm -rf /var/lib/apt/lists/* \
54-
&& cd /tmp \
55-
&& wget --no-check-certificate -O firefox-esr.tar.bz2 \
56-
'https://download.mozilla.org/?product=firefox-esr-latest&os=linux64&lang=en-US' \
57-
&& tar -xjf firefox-esr.tar.bz2 -C /opt/ \
58-
&& ln -s /opt/firefox/firefox /usr/bin/firefox \
59-
&& rm -f /tmp/firefox-esr.tar.bz2
36+
#======================
37+
# Install Chromedriver
38+
#======================
39+
RUN CHROMEDRIVER_VERSION=`curl -sS chromedriver.storage.googleapis.com/LATEST_RELEASE` && \
40+
mkdir -p /opt/chromedriver-$CHROMEDRIVER_VERSION && \
41+
curl -sS -o /tmp/chromedriver_linux64.zip http://chromedriver.storage.googleapis.com/$CHROMEDRIVER_VERSION/chromedriver_linux64.zip && \
42+
unzip -qq /tmp/chromedriver_linux64.zip -d /opt/chromedriver-$CHROMEDRIVER_VERSION && \
43+
rm /tmp/chromedriver_linux64.zip && \
44+
chmod +x /opt/chromedriver-$CHROMEDRIVER_VERSION/chromedriver && \
45+
ln -fs /opt/chromedriver-$CHROMEDRIVER_VERSION/chromedriver /usr/local/bin/chromedriver
6046

6147
#================
6248
# Install Chrome
@@ -67,22 +53,25 @@ RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key
6753
&& apt-get install -y google-chrome-stable \
6854
&& rm -rf /var/lib/apt/lists/*
6955

70-
#===================
71-
# Timezone settings
72-
#===================
73-
# Full list at http://en.wikipedia.org/wiki/List_of_tz_database_time_zones
74-
# e.g. "US/Pacific" for Los Angeles, California, USA
75-
ENV TZ "America/New_York"
76-
# Apply TimeZone
77-
RUN echo $TZ | tee /etc/timezone \
78-
&& dpkg-reconfigure --frontend noninteractive tzdata
56+
#==================
57+
# Configure Chrome
58+
#==================
59+
RUN dpkg-divert --add --rename --divert /opt/google/chrome/google-chrome.real /opt/google/chrome/google-chrome && \
60+
echo "#!/bin/bash\nexec /opt/google/chrome/google-chrome.real --disable-setuid-sandbox \"\$@\"" > /opt/google/chrome/google-chrome && \
61+
chmod 755 /opt/google/chrome/google-chrome
7962

80-
#========================================
81-
# Add normal user with passwordless sudo
82-
#========================================
83-
RUN sudo useradd seluser --shell /bin/bash --create-home \
84-
&& sudo usermod -a -G sudo seluser \
85-
&& echo 'ALL ALL = (ALL) NOPASSWD: ALL' >> /etc/sudoers
63+
#=================
64+
# Install Firefox
65+
#=================
66+
RUN apt-get -qy --no-install-recommends install \
67+
$(apt-cache depends firefox | grep Depends | sed "s/.*ends:\ //" | tr '\n' ' ') \
68+
&& rm -rf /var/lib/apt/lists/* \
69+
&& cd /tmp \
70+
&& wget --no-check-certificate -O firefox-esr.tar.bz2 \
71+
'https://download.mozilla.org/?product=firefox-esr-latest&os=linux64&lang=en-US' \
72+
&& tar -xjf firefox-esr.tar.bz2 -C /opt/ \
73+
&& ln -s /opt/firefox/firefox /usr/bin/firefox \
74+
&& rm -f /tmp/firefox-esr.tar.bz2
8675

8776
#===================
8877
# Install PhantomJS
@@ -93,6 +82,15 @@ RUN ln -s /usr/local/share/phantomjs-1.9.7-linux-x86_64/bin/phantomjs /usr/local
9382
RUN ln -s /usr/local/share/phantomjs-1.9.7-linux-x86_64/bin/phantomjs /usr/local/bin/phantomjs
9483
RUN ln -s /usr/local/share/phantomjs-1.9.7-linux-x86_64/bin/phantomjs /usr/bin/phantomjs
9584

85+
#===========================
86+
# Configure Virtual Display
87+
#===========================
88+
RUN set -e
89+
RUN echo "Starting X virtual framebuffer (Xvfb) in background..."
90+
RUN Xvfb -ac :99 -screen 0 1280x1024x16 > /dev/null 2>&1 &
91+
RUN export DISPLAY=:99
92+
RUN exec "$@"
93+
9694
#=====================
9795
# Set up SeleniumBase
9896
#=====================
@@ -103,13 +101,12 @@ COPY examples /SeleniumBase/examples/
103101
RUN cd /SeleniumBase && ls && sudo pip install -r docker_requirements.txt
104102
RUN cd /SeleniumBase && ls && sudo python docker_setup.py install
105103

106-
#=========================================
107-
# Create entrypoint and grab example test
108-
#=========================================
104+
#==========================================
105+
# Create entrypoint and grab example tests
106+
#==========================================
109107
COPY docker/docker-entrypoint.sh /
110108
COPY docker/run_docker_test_in_firefox.sh /
111109
COPY docker/run_docker_test_in_chrome.sh /
112-
COPY docker/run_docker_test_in_phantomjs.sh /
113110
COPY docker/docker_config.cfg /SeleniumBase/examples/
114111
ENTRYPOINT ["/docker-entrypoint.sh"]
115112
CMD ["/bin/bash"]

‎README.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,9 @@ If you're planning on using the full power of this test framework, there are a f
265265

266266
* Setup your Selenium Grid and update your *.cfg file to point there. An example config file called selenium_server_config_example.cfg has been provided for you in the grid folder. The start-selenium-node.bat and start-selenium-server.sh files are for running your grid. In an example situation, your Selenium Grid server might live on a unix box and your Selenium Grid nodes might live on EC2 Windows virtual machines. When your build server runs a Selenium test, it would connect to your Selenium Grid to find out which Grid browser nodes are available to run that test. To simplify things, you can use [Browser Stack](https://www.browserstack.com/automate) as your entire Selenium Grid (and let them do all the fun work of maintaining the grid for you).
267267

268-
* There are ways of running your tests from Jenkins without having to utilize a remote machine. One way is by using PhantomJS as your browser (it runs headlessly). Another way is by using Xvfb (another headless system). [There's a plugin for Xvfb in Jenkins](https://wiki.jenkins-ci.org/display/JENKINS/Xvfb+Plugin). Here are some more helpful resources I found regarding the use of Xvfb:
268+
* There are ways of running your tests from Jenkins without having to utilize a remote machine. One way is by using PhantomJS as your browser (it runs headlessly). Another way is by using Xvfb (another headless system). [There's a plugin for Xvfb in Jenkins](https://wiki.jenkins-ci.org/display/JENKINS/Xvfb+Plugin).
269+
If you have Xvfb running in the background, you can add ``--headless`` to your run command in order to utilize it.
270+
Here are some more helpful resources I found regarding the use of Xvfb:
269271
1. http://stackoverflow.com/questions/6183276/how-do-i-run-selenium-in-xvfb
270272
2. http://qxf2.com/blog/xvfb-plugin-for-jenkins-selenium/
271273
3. http://stackoverflow.com/questions/27202131/firefox-started-by-selenium-ignores-the-display-created-by-pyvirtualdisplay

‎conftest.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,11 @@ def pytest_addoption(parser):
3434
parser.addoption('--log_path', dest='log_path',
3535
default='logs/',
3636
help='Where the log files are saved.')
37+
parser.addoption('--headless', action="store_true",
38+
dest='headless',
39+
default=False,
40+
help="""Using this makes Webdriver run headlessly,
41+
which is useful inside a Linux Docker.""")
3742
parser.addoption('--demo_mode', action="store_true",
3843
dest='demo_mode',
3944
default=False,
@@ -50,6 +55,7 @@ def pytest_configure(config):
5055
with_testing_base = config.getoption('with_testing_base')
5156
browser = config.getoption('browser')
5257
log_path = config.getoption('log_path')
58+
headless = config.getoption('headless')
5359
demo_mode = config.getoption('demo_mode')
5460
demo_sleep = ''
5561
data = ''
@@ -65,6 +71,7 @@ def pytest_configure(config):
6571
config_file.write("data:::%s\n" % data)
6672
config_file.write("with_testing_base:::%s\n" % with_testing_base)
6773
config_file.write("log_path:::%s\n" % log_path)
74+
config_file.write("headless:::%s\n" % headless)
6875
config_file.write("demo_mode:::%s\n" % demo_mode)
6976
config_file.write("demo_sleep:::%s\n" % demo_sleep)
7077
config_file.close()

‎docker/ReadMe.md

Lines changed: 12 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -6,41 +6,41 @@
66

77
docker-machine create --driver virtualbox seleniumbase
88

9-
#### 3. If your Docker environment ever goes down for any reason, you can bring it back up with a restart:
9+
##### (If your Docker environment ever goes down for any reason, you can bring it back up with a restart.)
1010

1111
docker-machine restart seleniumbase
1212

13-
#### 4. Configure your shell:
13+
#### 3. Configure your shell:
1414

1515
eval "$(docker-machine env seleniumbase)"
1616

17-
#### 5. Go to the SeleniumBase home directory. (That's where "Dockerfile" is located)
17+
#### 4. Go to the SeleniumBase home directory. (That's where "Dockerfile" is located)
1818

19-
#### 6. Create your Docker image from your Dockerfile: (Get ready to wait awhile)
19+
#### 5. Create your Docker image from your Dockerfile: (Get ready to wait awhile)
2020

2121
docker build -t seleniumbase .
2222

23-
#### 7. Run a test inside your Docker: (Once the test completes after a few seconds, you'll automatically exit the Docker shell)
23+
#### 6. Run a test inside your Docker: (Once the test completes after a few seconds, you'll automatically exit the Docker shell)
2424

2525
docker run seleniumbase ./run_docker_test_in_firefox.sh
2626

27-
#### 8. Now run the same test with Chrome inside your Docker:
27+
#### 7. Now run the same test with Chrome inside your Docker:
2828

2929
docker run seleniumbase ./run_docker_test_in_chrome.sh
3030

31-
#### 9. You can also enter Docker and stay inside the shell:
31+
#### 8. You can also enter Docker and stay inside the shell:
3232

3333
docker run -i -t seleniumbase
3434

35-
#### 10. Now you can run the example test from inside the Docker shell: (This time with PhantomJS)
35+
#### 9. Now you can run the example test from inside the Docker shell:
3636

37-
./run_docker_test_in_phantomjs.sh
37+
./run_docker_test_in_chrome.sh
3838

39-
#### 11. When you're satisfied, you may exit the Docker shell:
39+
#### 10. When you're satisfied, you may exit the Docker shell:
4040

4141
exit
4242

43-
#### 12. (Optional) Since Docker images and containers take up a lot of space, you may want to clean up your machine from time to time when they’re not being used:
43+
#### 11. (Optional) Since Docker images and containers take up a lot of space, you may want to clean up your machine from time to time when they’re not being used:
4444
http://stackoverflow.com/questions/17236796/how-to-remove-old-docker-containers
4545
Here are a few of those cleanup commands:
4646

@@ -57,7 +57,7 @@ Finally, if you want to wipe out your SeleniumBase Docker virtualbox, use these
5757
docker-machine kill seleniumbase
5858
docker-machine rm seleniumbase
5959

60-
#### 13. (Optional) More reading on Docker can be found here:
60+
#### 12. (Optional) More reading on Docker can be found here:
6161
* https://docs.docker.com
6262
* https://docs.docker.com/mac/started/
6363
* https://docs.docker.com/installation/mac/

‎docker/docker_config.cfg

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
[nosetests]
2-
with-selenium_docker=1
2+
with-selenium=1
33
with-testing_base=1
44
with-basic_test_info=1
55
nocapture=1

‎docker/docker_setup.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88

99
setup(
1010
name='seleniumbase',
11-
version='1.1.17',
11+
version='1.1.18',
1212
author='Michael Mintz',
1313
author_email='@mintzworld',
1414
maintainer='Michael Mintz',
@@ -24,8 +24,7 @@
2424
entry_points={
2525
'nose.plugins': [
2626
'base_plugin = seleniumbase.plugins.base_plugin:Base',
27-
'selenium_docker = '
28-
'seleniumbase.plugins.docker_selenium_plugin:SeleniumBrowser',
27+
'selenium = seleniumbase.plugins.selenium_plugin:SeleniumBrowser',
2928
'page_source = seleniumbase.plugins.page_source:PageSource',
3029
'screen_shots = seleniumbase.plugins.screen_shots:ScreenShots',
3130
'test_info = seleniumbase.plugins.basic_test_info:BasicTestInfo',

‎docker/run_docker_test_in_chrome.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22
set -e
33
# Run example test from inside Docker image
44
echo "Running example SeleniumBase test from Docker with headless Chrome..."
5-
cd /SeleniumBase/examples/ && nosetests my_first_test.py --config=docker_config.cfg --browser=chrome
5+
cd /SeleniumBase/examples/ && nosetests my_first_test.py --config=docker_config.cfg --browser=chrome --headless
66
exec "$@"

‎docker/run_docker_test_in_firefox.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,5 +2,5 @@
22
set -e
33
# Run example test from inside Docker image
44
echo "Running example SeleniumBase test from Docker with headless Firefox..."
5-
cd /SeleniumBase/examples/ && nosetests my_first_test.py --config=docker_config.cfg --browser=firefox
5+
cd /SeleniumBase/examples/ && nosetests my_first_test.py --config=docker_config.cfg --browser=firefox --headless
66
exec "$@"

‎docker/run_docker_test_in_phantomjs.sh

Lines changed: 0 additions & 6 deletions
This file was deleted.

‎requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,5 @@ simplejson==3.8.1
1111
boto==2.38.0
1212
pdb==0.1
1313
ipdb==0.8.1
14+
pyvirtualdisplay==0.1.5
1415
-e .

0 commit comments

Comments
 (0)