Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Appearance settings

Commit 2c124b7

Browse filesBrowse files
committed
Merge branch '71-data-retrieval-dump' into 'master'
feat: support logical dumps (internal#71) Create a new job `logical-dump` to support logical dumps including: * logical dump from local or remote Postgres instance; * direct dump/restore process without having an extra copy; The known limitation is a single thread for the direct restoring. See merge request postgres-ai/database-lab!125
2 parents f0adf80 + 52176c5 commit 2c124b7
Copy full SHA for 2c124b7

File tree

9 files changed

+487
-77
lines changed
Filter options

9 files changed

+487
-77
lines changed

‎.gitignore

Copy file name to clipboardExpand all lines: .gitignore
-1Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,4 +8,3 @@
88
/deploy/
99

1010
/configs/config.yml
11-
/configs/retrieval.yml

‎configs/config.sample.yml

Copy file name to clipboardExpand all lines: configs/config.sample.yml
+51-8Lines changed: 51 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -92,16 +92,59 @@ retrieval:
9292
- initialize
9393

9494
spec:
95+
# The initialize stage provides declarative initialization of the PostgreSQL data directory used by Database Lab Engine.
96+
# The stage must not contain physical and logical restore jobs simultaneously.
9597
initialize:
9698
jobs:
97-
# - name: logical-restore
98-
# options:
99-
# dumpFile: /tmp/db.dump
100-
# forceInit: false
101-
# dbName: test
102-
# partial:
103-
# tables:
104-
# - test
99+
# Dumps PostgreSQL database from provided source.
100+
- name: logical-dump
101+
options:
102+
# The dump file will be automatically created on this location and then used to restore.
103+
dumpLocation: /tmp/db.dump
104+
105+
# The Docker image containing the tools required to get a dump.
106+
dockerImage: "postgresai/retrieval:12"
107+
108+
# Connection parameters of the database to be dumped.
109+
connection:
110+
type: local
111+
dbname: postgres
112+
host: 127.0.0.1
113+
port: 5432
114+
username: postgres
115+
password: postgres # The environment variable PGPASSWORD can be used instead of this option.
116+
117+
# Options for a partial dump.
118+
partial:
119+
tables:
120+
- test
121+
122+
# The number of parallel jobs to get a dump.
123+
parallelJobs: 1
124+
125+
# Options for direct restore to Database Lab Engine instance.
126+
restore:
127+
# Restore data even if the Postgres directory (`global.dataDir`) is not empty.
128+
# Note the existing data might be overwritten.
129+
forceInit: true
130+
131+
# Restores PostgreSQL database from the provided dump.
132+
- name: logical-restore
133+
options:
134+
dbname: test
135+
# The location of the archive file (or directory, for a directory-format archive) to be restored.
136+
dumpLocation: /tmp/db.dump
137+
138+
# Restore data even if the Postgres directory (`global.dataDir`) is not empty.
139+
# Note the existing data might be overwritten.
140+
forceInit: false
141+
142+
# Options for a partial dump.
143+
partial:
144+
tables:
145+
- test
146+
147+
# Restores database data from a physical backup.
105148
- name: physical-restore
106149
options:
107150
tool: walg

‎pkg/config/config.go

Copy file name to clipboardExpand all lines: pkg/config/config.go
+2-3Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,9 +34,8 @@ type Config struct {
3434

3535
// Global contains global Database Lab configurations.
3636
type Global struct {
37-
Engine string `yaml:"engine"`
38-
DataDir string `yaml:"dataDir"`
39-
DockerImage string `yaml:"dockerImage"`
37+
Engine string `yaml:"engine"`
38+
DataDir string `yaml:"dataDir"`
4039
}
4140

4241
// LoadConfig instances a new Config by configuration filename.

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.