Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 807cd7f

Browse filesBrowse files
committed
Merge pull request metacpan#4 from CPAN-API/cpan-api
Dockerfile & README.md for cpan-api
2 parents ff5c7d2 + da60574 commit 807cd7f
Copy full SHA for 807cd7f

File tree

Expand file treeCollapse file tree

6 files changed

+273
-3
lines changed
Filter options
Expand file treeCollapse file tree

6 files changed

+273
-3
lines changed

‎README.md

Copy file name to clipboard
+159-3Lines changed: 159 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,160 @@
1-
metacpan-docker
2-
===============
1+
# Running full metacpan stack with docker
32

4-
Experimental docker configs for metacpan
3+
## Notice
4+
5+
This project is in experimental stage. It works, but there are a lot of things
6+
to be done better. Please use it and create Issues with your problems.
7+
8+
## Installation
9+
10+
* [Install Docker](https://docs.docker.com/installation/)
11+
* Clone this repo
12+
* Build all images with the comands like:
13+
`cd cpan-api; docker build --tag cpan-api .`
14+
15+
## System architecture
16+
17+
The system consists of serverl microservices that live in docker containes:
18+
19+
* `cpan_volume` — data volume container that shares directory `/cpan` with
20+
all other containers
21+
* `elasticsearch` — database
22+
* `cpan-api` — the main server — it uses `elasticsearch` and `/cpan`
23+
directory
24+
* `metacpan-web` — the web interface — it works with `cpan-api`
25+
26+
## cpan_volume
27+
28+
Eveything starts from the volume `/cpan`.
29+
30+
First you need to create data volume contaner:
31+
32+
docker run --name cpan_volume --volume=/cpan ubuntu:14.04
33+
34+
Then it is possible to add some files to that volume. The simplest way is to
35+
use `orepan2` image. But there can be other ways to populate the `/cpan`
36+
volume. Here is a sample command that adds module from the big cpan to your
37+
`/cpan` volume:
38+
39+
docker run \
40+
--rm \
41+
--volumes-from=cpan_volume \
42+
orepan2 \
43+
orepan2-inject --author BESSARABV App::Stopwatch /cpan
44+
45+
One can inspect the content of the volume with one-shot container:
46+
47+
docker run \
48+
--rm \
49+
--volumes-from=cpan_volume \
50+
ubuntu:14.04 \
51+
find /cpan
52+
53+
The output will be something like:
54+
55+
/cpan
56+
/cpan/modules
57+
/cpan/modules/02packages.details.txt.gz
58+
/cpan/orepan2-cache.json
59+
/cpan/authors
60+
/cpan/authors/id
61+
/cpan/authors/id/B
62+
/cpan/authors/id/B/BE
63+
/cpan/authors/id/B/BE/BESSARABV
64+
/cpan/authors/id/B/BE/BESSARABV/App-Stopwatch-1.2.0.tar.gz
65+
66+
You also need to generate `00whois.xml` file. If you use logins from big cpan
67+
you can get that file from cpan:
68+
69+
docker run \
70+
--rm \
71+
--volumes-from=cpan_volume \
72+
orepan2 \
73+
curl -o /cpan/authors/00whois.xml cpan.cpantesters.org/authors/00whois.xml
74+
75+
After we have some data in `/cpan` it is possible to add webinterface to it.
76+
77+
## elasticsearch
78+
79+
First you need to run container with elasticsearch:
80+
81+
docker run \
82+
--detach \
83+
--publish 9200:9200 \
84+
--name elasticsearch \
85+
elasticsearch
86+
87+
You can check that you have elasticsearch running with the comand:
88+
89+
curl 127.0.0.1:9200
90+
91+
PS If you run docker on mac or windows you should change `127.0.0.1` to the ip
92+
address of our docker virtual machinge (you can find out this ip with the
93+
`boot2docker ip`).
94+
95+
Here is the output you are expecred to see:
96+
97+
{
98+
"ok" : true,
99+
"status" : 200,
100+
"name" : "Cage, Luke",
101+
"version" : {
102+
"number" : "0.90.7",
103+
"build_hash" : "36897d07dadcb70886db7f149e645ed3d44eb5f2",
104+
"build_timestamp" : "2013-11-13T12:06:54Z",
105+
"build_snapshot" : false,
106+
"lucene_version" : "4.5.1"
107+
},
108+
"tagline" : "You Know, for Search"
109+
}
110+
111+
## cpan-api
112+
113+
Next you need to run cpan-api server. This can be done with the command:
114+
115+
docker run \
116+
--detach \
117+
--volumes-from=cpan_volume \
118+
--link=elasticsearch:elasticsearch \
119+
--volume=$(pwd)/configs/cpan-api/metacpan.pl:/cpan-api/etc/metacpan.pl \
120+
--volume=$(pwd)/configs/cpan-api/metacpan_server.conf:/cpan-api/metacpan_server.conf \
121+
--env MINICPAN=/cpan \
122+
--publish 5000:5000 \
123+
--name cpan-api \
124+
cpan-api
125+
126+
So the server is running but you also need to run some scripts to index data.
127+
To do it you can create ineractive container:
128+
129+
docker run \
130+
-it \
131+
--rm \
132+
--volumes-from=cpan_volume \
133+
--link=elasticsearch:elasticsearch \
134+
--volume=$(pwd)/configs/cpan-api/metacpan.pl:/cpan-api/etc/metacpan.pl \
135+
--volume=$(pwd)/configs/cpan-api/metacpan_server.conf:/cpan-api/metacpan_server.conf \
136+
--env MINICPAN=/cpan \
137+
cpan-api \
138+
bash
139+
140+
And then execute all the needed scripts:
141+
142+
carton exec bin/metacpan mapping --delete
143+
carton exec bin/metacpan release /cpan/authors/id/
144+
carton exec bin/metacpan latest --cpan /cpan/
145+
carton exec bin/metacpan author --cpan /cpan/
146+
147+
## metacpan-web
148+
149+
Then you need to run metacpan-web:
150+
151+
docker run \
152+
--detach \
153+
--publish 5001:5001 \
154+
--link=cpan-api:cpan-api \
155+
--volume=$(pwd)/configs/metacpan-web/metacpan_web.conf:/root/metacpan-web/metacpan_web.conf \
156+
--name metacpan-web \
157+
metacpan-web
158+
159+
Open your browser at http://127.0.0.1:5001 and you will see metacpan web
160+
interface.

‎configs/cpan-api/metacpan.pl

Copy file name to clipboard
+23Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# do not edit this file
2+
# create etc/metacpan_local.pl instead
3+
use FindBin;
4+
5+
{
6+
# ElasticSearch instance, can be either a single server
7+
# or an arrayref of servers
8+
es => 'elasticsearch:9200',
9+
# the port of the api server
10+
port => '5000',
11+
# log level
12+
level => 'info',
13+
# appender for Log4perl
14+
# default layout is "%d %p{1} %c: %m{chomp}%n"
15+
# can be overridden using the layout key
16+
# defining logger in metacpan_local.pl will
17+
# override and not append to this configuration
18+
logger => [{
19+
class => 'Log::Log4perl::Appender::File',
20+
filename => $FindBin::RealBin . '/../var/log/metacpan.log',
21+
syswrite => 1,
22+
}]
23+
}

‎configs/cpan-api/metacpan_server.conf

Copy file name to clipboard
+19Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
git /usr/bin/git
2+
cpan /cpan
3+
4+
<model CPAN>
5+
servers elasticsearch:9200
6+
</model CPAN>
7+
8+
<model User>
9+
servers elasticsearch:9200
10+
</model User>
11+
12+
<plugin Session>
13+
servers elasticsearch:9200
14+
</plugin>
15+
16+
<controller User::Turing>
17+
# required for server startup -- override this in metacpan_server_local.conf
18+
private_key 59125ffc09413eed3f2a2c07a37c7a44b95633e2
19+
</controller>
+38Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# rename this file to metacpan::web.yml and put a ':' after 'name' if
2+
# you want to use YAML like in old versions of Catalyst
3+
name MetaCPAN::Web
4+
default_view HTML
5+
6+
api = http://cpan-api:5000
7+
api_external = http://cpan-api:5000
8+
api_secure = http://cpan-api:5000
9+
api_external_secure = http://cpan-api:5000
10+
consumer_key = metacpan.dev
11+
cookie_secret = seekrit
12+
consumer_secret = ClearAirTurbulence
13+
14+
mark_unauthorized_releases = 0
15+
16+
#site_alert_message = The sky is falling.
17+
18+
<view HTML>
19+
INCLUDE_PATH root/
20+
TAG_STYLE asp
21+
PRE_PROCESS preprocess.html
22+
WRAPPER wrapper.html
23+
TEMPLATE_EXTENSION .html
24+
ENCODING utf8
25+
AUTO_FILTER html
26+
STAT_TTL 1
27+
COMPILE_PERL 2
28+
COMPILE_DIR var/tmp/templates
29+
</view>
30+
31+
<view Raw>
32+
WRAPPER ""
33+
</view>
34+
35+
<controller Account::Turing>
36+
public_key 6LeH2MsSAAAAANwz3AA73Gw5OjCVjT6I51Ev-ior
37+
</controller>
38+

‎cpan-api/Dockerfile

Copy file name to clipboard
+26Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
FROM ubuntu:14.04
2+
3+
ENV UPDATED_AT 2014-11-17
4+
5+
RUN apt-get update
6+
7+
RUN apt-get install -y \
8+
curl \
9+
gcc \
10+
git \
11+
libcurl4-openssl-dev \
12+
libexpat1-dev \
13+
libxml2-dev \
14+
make
15+
16+
RUN curl -L http://cpanmin.us | perl - App::cpanminus
17+
18+
RUN cpanm Carton
19+
20+
RUN git clone https://github.com/CPAN-API/cpan-api.git /cpan-api/
21+
22+
WORKDIR /cpan-api/
23+
24+
RUN carton install
25+
26+
CMD ["carton", "exec", "plackup", "-p", "5000", "-r"]

‎cpan-api/README.md

Copy file name to clipboard
+8Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
2+
You can build a docker image with command:
3+
4+
docker build --tag cpan-api .
5+
6+
And then run it:
7+
8+
docker run -it cpan-api /bin/bash

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.