Setup and Configuration
This tutorial is the second part of a three part series of blog posts where we walk you through the process in which to successfully deploy the default UC Berkeley CollectionSpace Django web application on a production server. The first part, ‘’Installing and Configuring the Dependencies,” can be found here. In this tutorial we will perform the following in order to successfully deploy our Django web application:
- Getting the source code
- Setting up our Python virtual environment
- Editing the wsgi.py script
- Collecting the static files
- Configuring the Apache2 web server
- Deploying and troubleshooting
- Next steps
Before we get started, let’s recap on CollectionSpace, the Django web application, and what was covered in the preceding tutorial.
CollectionSpace is an open-source, community based, web application built for museum and collection institutions to manage their internal collections. You can learn more about CollectionSpace by visiting www.collectionspace.org.
The CollectionSpace Django web application is also open-source and community based, but represents a public facing site with many available internal applications such as:
- public search and reporting,
- internal search and reporting,
- collection browsing,
- image browsing,
- bulk Media uploading,
- batch image uploading,
- and more.
To preview several deployments of the Django web application, navigate to webapps.cspace.berkeley.edu. In addition, the source code can be found at github.com/cspace-deployment/cspace_django_project.
In part one of this tutorial, we installed and configured the various dependencies needed to run the Django web application in a production environment, these included:
- Installing the Apache2 web server to run as a service on your Ubuntu 14.04 machine.
- Installing Apache Solr, a powerful enterprise search server, with a “REST-like” API, to also run as a service.
- Ensuring Python, Python’s package manager, PIP and Virtual Environment were installed on your machine, and if not, providing resources to do so.
Now that we have all of the background thoroughly covered, let’s begin with part two of this series. Our assumptions are that you’ve successfully completed the preceding tutorial and you currently have Solr and Apache2 running as a service on your machine. If not, feel free to contact us for help or post on the CollectionSpace Talk list.
Getting the source code
Navigate to the shared directory at /usr/local/share or more specifically:
$ cd /usr/local/share/
From within the above directory, make a new directory called “django” and within this django directory, a new subdirectory called “webapp” by executing:
$ sudo mkdir -p django/webapp
Next, navigate into the webapp directory:
$ cd /usr/local/share/django/webapp/
Now, get the source code by cloning the remote GitHub repository, locally:
$ sudo git clone https://github.com/cspace-deployment/cspace_django_project.git
On the off chance that you may not have ‘git’ installed on your machine, run:
$ sudo apt-get install git
That should do it, as far as obtaining the latest and greatest source code.
Now, copy all of the files found in the ‘cspace_django_project’ directory to the parent directory, ‘webapp’, by executing, from within the ‘webapp’ directory:
$ sudo cp -r cspace_django_project/* .
Then, after effectively copying all the files, remove the ‘cspace_django_directory’ by executing:
$ sudo rm -r cspace_django_directory
Setting up our Python virtual environment
Now that we have the source code, we need to ensure that the dependencies required for this project remain separate of other Python projects on our machine. This is accomplished by isolating our project using the Virtual Environment tool, which was installed in part of this series. However, in order to begin, we still need to set some path variables for our virtual environments. To do so, execute the following commands, from within the webapp directory:
Note: although it’s possible to use the virtual environment wrapper (virtualenvwrapper) tool, you are likely to run into various permission issues that can easily be avoided by using the simple virtual environment tool (virtualenv).
$ sudo su
Warning: we are now the root user of the machine, be careful and mindful of what you do as the root user.
Now, create a virtual environment for our project, while still in the webapp directory, by running:
# virtualenv cspace_venv
Note: the ’#’ indicates that the current user is the root user. And (cspace_venv) indicates the current working virtual environment.
We are now ready to begin working in our dedicated virtual environment, ‘cspace_venv.’ You may now see the prefix ‘(cspace_venv)’ on the current line, defining the current working virtual environment. If not, run:
# source cspace_venv/bin/activate
A couple of useful commands to take note of:
(cspace_venv) # deactivate
Deactivates the current working virtual environment.
And,
# rm -rf [name of the virtual environment to remove]
Which will remove any defined virtual environment, by simply deleting its directory.
Moving on, install the web application’s requirements within ‘cspace_venv’ by executing:
(cspace_venv) # pip install -r pycharm_requirements.txt
As suggested in the project readme, the dependencies found in the pycharm_requirements.txt are the least resource intensive, and simplest to install.
If that fails with something like “Command ‘python setup.py egg_info’ failed with error code 1 in /tmp/pip-build-dUM8oT/PyGreSQL/”, try the following:
# apt-get install python-psycopg2
# apt-get install libpq-dev
(cspace_venv) # pip install -r pycharm_requirements.txt
The dependency requirements should now install fully, and you should be prompted with “successfully installed Pillow-2.5.3 PyGreSQL-4.1.1 …..” If not, we would suggest searching for the error you may receive and posting to the CollectionSpace Talk list.
Next, install the Apache mod_wsgi module by executing:
# apt-get install libapache2-mod-wsgi
This module is used to host Python standard WSGI applications on an Apache server. You can read more about it here: https://docs.djangoproject.com/en/1.9/howto/deployment/wsgi/modwsgi/
The next step is to ensure that all of the configuration files are in the right place for our deployment. Example configuration files can be found in the ‘config.examples’ directory. For a default deployment on a production server, these configuration files are adequate. Therefore, will will need to move them to the ‘config’ directory in order to satisfy the applications configuration requirements. To do so, from within the ‘webapp’ directory, execute the command’:
# cp -r config.examples/*.cfg config
(cspace_venv) # ./setup.sh configure pycharm
(cspace_venv) # ./setup.sh deploy default
You will likely see a bunch of warnings, ignore them for now.
Editing the wsgi.py script
We’ll need to make some additions to the wsgi.py script found in the ‘django/webapp/cspace_django_site’ directory in order to activate and point to the dedicated virtual environment, containing module dependencies we installed earlier, upon deployment.
Using your preferred command line text editing tool, add the following statements and expressions at the specified line numbers. We are not removing any lines, just adding them:
1 import site
.
.
11 # Activate virtual environment
12 activate_env = os.path.expanduser('/usr/local/share/django/webapp/cspace_venv/bin/activate_this.py')
13 execfile(activate_env, dict(__file__=activate_env))
Configuring the Apache2 server
Configure the Apache2 server to communicate with the Django web application by copying the provided file 000-default.conf found in ‘django/webapp/config.examples’ and overriding Apache2’s default configuration file. However, before doing so, following the instructions in the provided configuration file and then ensure that lines 50-52 are the following, adding them if necessary:
50 WSGIDaemonProcess webapp_wsgi python-path=/usr/local/share/django/webapp:/usr/local/share/django/webapp/cspace_venv/lib/python2.7/site-packages
51 WSGIProcessGroup webapp_wsgi
52 WSGIScriptAlias /webapp /usr/local/share/django/webapp/cspace_django_site/wsgi.py process-group=webapp_wsgi
Do not remove any lines, just comment them out to be safe.
Then, save your edits and copy the provided configuration file to the Apache2 directory by executing:
# cp -f ./000-default.conf /etc/apache2/site-enabled/
Collecting the static files for the webapp
Excellent, now collect the static files by executing, from within the ‘webapp’ directory:
(cspace_venv)# python manage.py collectstatic
Note: make sure you are still working in the ‘cspace_venv’ dedicated virtual environment, otherwise you will receive errors.
Type ‘yes’, effectively collecting all of the static files in STATIC_ROOT.
Then, restart the Apache2 server:
# service apache2 restart
Deploying and Troubleshooting
Visit ‘<your_ip_address>/webapp’ in your browser to ensure the webapp was deployed successfully. If all went well, you should be seeing:
And, clicking the ‘search’ button, effectively directing us to the search application, you should see:
However, if you run into errors, check the Apache2 server log located at /var/logs/apache2/error.log by executing:
# tail -f /var/logs/apache2/error.log
Back in the ‘webapp’ directory, you may need to disable many of the pre-installed web applications by first, listing the currently installed apps:
(cspace_venv)# ./setup show
And, disabling any that may be causing problems, for example:
(cspace_venv)# ./setup disable hello
Next steps
If all went well, you should now be able to access and view your new UCB CollectionSpace Django web application from your browser, including the the public search application. If not, continue to troubleshoot by using the Apache2 error.log, and as always, feel free to post any difficulties on the CollectionSpace Talk list.
Okay, so we are not completely out of the woods yet. You’ll notice that any queries performed in the search application returns no list of records. Now you’re likely asking, “Yousuf, what gives?”
Well, this is due to the fact that we have yet to configure Solr to talk to our CollectionSpace PostgreSQL database. That, we will cover in the next post in this series, along with:
- understanding the Solr tools, scripts, and files built by UCB,
- configuring the imageserver so that images are provided in search results,
- and next steps.
As always, we hope you found this tutorial useful and we look forward to seeing you as part of the growing CollectionSpace community. For questions about how CollectionSpace can help manage your museum collections, feel free to contact us and our development team here at Granite Horizon.
