Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly
Appearance settings

Commit 7de1599

Browse filesBrowse files
cuiy0006cuiy0006
authored
feat(dataform): generate library (googleapis#15151)
* feat(dataform): generate library * Run generators and format their outputs * Add API baseline * Manually update READMEs, quickstart, and top-level stuff * doc link
1 parent f813cb6 commit 7de1599
Copy full SHA for 7de1599

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Dismiss banner
Expand file treeCollapse file tree

57 files changed

+14863
-0
lines changed

‎CHANGELOG.md

Copy file name to clipboardExpand all lines: CHANGELOG.md
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ the APIs in these libraries are stable, and are ready for production use.
1313

1414
- [Memorystore API](/google/cloud/memorystore/README.md)
1515
- [Oracle Database@Google Cloud API](/google/cloud/oracledatabase/README.md)
16+
- [Dataform API](/google/cloud/dataform/README.md)
1617

1718
## v2.37.0 - 2025-05
1819

‎README.md

Copy file name to clipboardExpand all lines: README.md
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,9 @@ See each library's `README.md` file for more information about:
203203
- [Google Cloud Data Catalog API](google/cloud/datacatalog/README.md)
204204
[[quickstart]](google/cloud/datacatalog/quickstart/README.md)
205205
[[reference]](https://cloud.google.com/cpp/docs/reference/datacatalog/latest)
206+
- [Dataform API](google/cloud/dataform/README.md)
207+
[[quickstart]](google/cloud/dataform/quickstart/README.md)
208+
[[reference]](https://cloud.google.com/cpp/docs/reference/dataform/latest)
206209
- [Cloud Data Fusion API](google/cloud/datafusion/README.md)
207210
[[quickstart]](google/cloud/datafusion/quickstart/README.md)
208211
[[reference]](https://cloud.google.com/cpp/docs/reference/datafusion/latest)

‎ci/abi-dumps/google_cloud_cpp_dataform.expected.abi.dump.gz

Copy file name to clipboard
668 KB
Binary file not shown.

‎ci/cloudbuild/builds/cmake-install.sh

Copy file name to clipboardExpand all lines: ci/cloudbuild/builds/cmake-install.sh
+3Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,9 @@ expected_dirs+=(
9191
# no RPC services in google/cloud/compute common proto dirs
9292
./include/google/cloud/compute/v1
9393
./include/google/cloud/compute/v1/internal
94+
# no RPC services in google/cloud/dataform/logging
95+
./include/google/cloud/dataform/logging
96+
./include/google/cloud/dataform/logging/v1
9497
./include/google/cloud/gkebackup/logging
9598
./include/google/cloud/gkebackup/logging/v1
9699
./include/google/cloud/gkehub/v1/configmanagement

‎ci/etc/expected_install_directories

Copy file name to clipboardExpand all lines: ci/etc/expected_install_directories
+6Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -665,6 +665,10 @@
665665
./include/google/cloud/datacatalog/v1
666666
./include/google/cloud/datacatalog/v1/internal
667667
./include/google/cloud/datacatalog/v1/mocks
668+
./include/google/cloud/dataform
669+
./include/google/cloud/dataform/v1
670+
./include/google/cloud/dataform/v1/internal
671+
./include/google/cloud/dataform/v1/mocks
668672
./include/google/cloud/datafusion
669673
./include/google/cloud/datafusion/v1
670674
./include/google/cloud/datafusion/v1/internal
@@ -1378,6 +1382,8 @@
13781382
./lib64/cmake/google_cloud_cpp_contentwarehouse_mocks
13791383
./lib64/cmake/google_cloud_cpp_datacatalog
13801384
./lib64/cmake/google_cloud_cpp_datacatalog_mocks
1385+
./lib64/cmake/google_cloud_cpp_dataform
1386+
./lib64/cmake/google_cloud_cpp_dataform_mocks
13811387
./lib64/cmake/google_cloud_cpp_datafusion
13821388
./lib64/cmake/google_cloud_cpp_datafusion_mocks
13831389
./lib64/cmake/google_cloud_cpp_datamigration

‎cmake/GoogleCloudCppFeatures.cmake

Copy file name to clipboardExpand all lines: cmake/GoogleCloudCppFeatures.cmake
+1Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,7 @@ set(GOOGLE_CLOUD_CPP_GA_LIBRARIES
8181
"containeranalysis"
8282
"contentwarehouse"
8383
"datacatalog"
84+
"dataform"
8485
"datafusion"
8586
"datamigration"
8687
"dataplex"

‎external/googleapis/protodeps/dataform.deps

Copy file name to clipboard
+8Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
@com_google_googleapis//google/api:annotations_proto
2+
@com_google_googleapis//google/api:client_proto
3+
@com_google_googleapis//google/api:field_behavior_proto
4+
@com_google_googleapis//google/api:http_proto
5+
@com_google_googleapis//google/api:launch_stage_proto
6+
@com_google_googleapis//google/api:resource_proto
7+
@com_google_googleapis//google/rpc:status_proto
8+
@com_google_googleapis//google/type:interval_proto

‎external/googleapis/protolists/dataform.list

Copy file name to clipboard
+2Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
@com_google_googleapis//google/cloud/dataform/logging/v1:logging.proto
2+
@com_google_googleapis//google/cloud/dataform/v1:dataform.proto

‎external/googleapis/update_libraries.sh

Copy file name to clipboardExpand all lines: external/googleapis/update_libraries.sh
+5Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,11 @@ declare -A -r LIBRARIES=(
114114
"@com_google_googleapis//google/cloud/datacatalog/v1:datacatalog_cc_grpc" \
115115
"@com_google_googleapis//google/cloud/datacatalog/lineage/v1:lineage_cc_grpc"
116116
)"
117+
["dataform"]="$(
118+
printf ",%s" \
119+
"@com_google_googleapis//google/cloud/dataform/v1:dataform_cc_grpc" \
120+
"@com_google_googleapis//google/cloud/dataform/logging/v1:logging_cc_grpc"
121+
)"
117122
["datafusion"]="@com_google_googleapis//google/cloud/datafusion/v1:datafusion_cc_grpc"
118123
["datamigration"]="$(
119124
printf ",%s" \

‎generator/generator_config.textproto

Copy file name to clipboardExpand all lines: generator/generator_config.textproto
+8Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1833,6 +1833,14 @@ service {
18331833
retryable_status_codes: ["kUnavailable"]
18341834
}
18351835
1836+
# Data Form
1837+
service {
1838+
service_proto_path: "google/cloud/dataform/v1/dataform.proto"
1839+
product_path: "google/cloud/dataform/v1"
1840+
initial_copyright_year: "2025"
1841+
retryable_status_codes: ["kUnavailable"]
1842+
}
1843+
18361844
# Data Catalog
18371845
service {
18381846
service_proto_path: "google/cloud/datacatalog/v1/datacatalog.proto"

‎google/cloud/dataform/BUILD.bazel

Copy file name to clipboard
+31Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,31 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# https://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
15+
load("//bazel:gapic.bzl", "cc_gapic_library")
16+
17+
package(default_visibility = ["//visibility:private"])
18+
19+
licenses(["notice"]) # Apache 2.0
20+
21+
service_dirs = ["v1/"]
22+
23+
googleapis_deps = [
24+
"@com_google_googleapis//google/cloud/dataform/v1:dataform_cc_grpc",
25+
]
26+
27+
cc_gapic_library(
28+
name = "dataform",
29+
googleapis_deps = googleapis_deps,
30+
service_dirs = service_dirs,
31+
)

‎google/cloud/dataform/CMakeLists.txt

Copy file name to clipboard
+34Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# ~~~
2+
# Copyright 2025 Google LLC
3+
#
4+
# Licensed under the Apache License, Version 2.0 (the "License");
5+
# you may not use this file except in compliance with the License.
6+
# You may obtain a copy of the License at
7+
#
8+
# https://www.apache.org/licenses/LICENSE-2.0
9+
#
10+
# Unless required by applicable law or agreed to in writing, software
11+
# distributed under the License is distributed on an "AS IS" BASIS,
12+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
# See the License for the specific language governing permissions and
14+
# limitations under the License.
15+
# ~~~
16+
17+
include(GoogleCloudCppLibrary)
18+
19+
google_cloud_cpp_add_gapic_library(dataform "Dataform API" SERVICE_DIRS "v1/")
20+
21+
if (BUILD_TESTING AND GOOGLE_CLOUD_CPP_ENABLE_CXX_EXCEPTIONS)
22+
add_executable(dataform_quickstart "quickstart/quickstart.cc")
23+
target_link_libraries(dataform_quickstart
24+
PRIVATE google-cloud-cpp::dataform)
25+
google_cloud_cpp_add_common_options(dataform_quickstart)
26+
add_test(
27+
NAME dataform_quickstart
28+
COMMAND
29+
cmake -P "${PROJECT_SOURCE_DIR}/cmake/quickstart-runner.cmake"
30+
$<TARGET_FILE:dataform_quickstart> GOOGLE_CLOUD_PROJECT
31+
GOOGLE_CLOUD_CPP_TEST_REGION)
32+
set_tests_properties(dataform_quickstart
33+
PROPERTIES LABELS "integration-test;quickstart")
34+
endif ()

‎google/cloud/dataform/README.md

Copy file name to clipboard
+60Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Dataform API C++ Client Library
2+
3+
This directory contains an idiomatic C++ client library for the \[Dataform
4+
API\][cloud-service-docs].
5+
6+
Service to develop, version control, and operationalize SQL pipelines in
7+
BigQuery.
8+
9+
While this library is **GA**, please note that the Google Cloud C++ client
10+
libraries do **not** follow [Semantic Versioning](https://semver.org/).
11+
12+
## Quickstart
13+
14+
The [quickstart/](quickstart/README.md) directory contains a minimal environment
15+
to get started using this client library in a larger project. The following
16+
"Hello World" program is used in this quickstart, and should give you a taste of
17+
this library.
18+
19+
<!-- inject-quickstart-start -->
20+
21+
```cc
22+
#include "google/cloud/dataform/v1/dataform_client.h"
23+
#include "google/cloud/location.h"
24+
#include <iostream>
25+
26+
int main(int argc, char* argv[]) try {
27+
if (argc != 3) {
28+
std::cerr << "Usage: " << argv[0] << " project-id location-id\n";
29+
return 1;
30+
}
31+
32+
auto const location = google::cloud::Location(argv[1], argv[2]);
33+
34+
namespace dataform = ::google::cloud::dataform_v1;
35+
auto client = dataform::DataformClient(dataform::MakeDataformConnection());
36+
37+
for (auto r : client.ListRepositories(location.FullName())) {
38+
if (!r) throw std::move(r).status();
39+
std::cout << r->DebugString() << "\n";
40+
}
41+
42+
return 0;
43+
} catch (google::cloud::Status const& status) {
44+
std::cerr << "google::cloud::Status thrown: " << status << "\n";
45+
return 1;
46+
}
47+
```
48+
49+
<!-- inject-quickstart-end -->
50+
51+
## More Information
52+
53+
- Official documentation about the [Dataform API][cloud-service-docs] service
54+
- [Reference doxygen documentation][doxygen-link] for each release of this
55+
client library
56+
- Detailed header comments in our \[public `.h`\][source-link] files
57+
58+
\[cloud-service-docs\]: https://cloud.google.com/dataform/docs \[doxygen-link\]:
59+
https://cloud.google.com/cpp/docs/reference/dataform/latest/ \[source-link\]:
60+
https://github.com/googleapis/google-cloud-cpp/tree/main/google/cloud/dataform

‎google/cloud/dataform/doc/environment-variables.dox

Copy file name to clipboard
+50Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
/*!
2+
3+
@page dataform-env Environment Variables
4+
5+
A number of environment variables can be used to configure the behavior of
6+
the library. There are also functions to configure this behavior in code. The
7+
environment variables are convenient when troubleshooting problems.
8+
9+
@section dataform-env-endpoint Endpoint Overrides
10+
11+
<!-- inject-endpoint-env-vars-start -->
12+
13+
- `GOOGLE_CLOUD_CPP_DATAFORM_ENDPOINT=...` overrides the
14+
`EndpointOption` (which defaults to "dataform.googleapis.com")
15+
used by `MakeDataformConnection()`.
16+
17+
<!-- inject-endpoint-env-vars-end -->
18+
19+
@see google::cloud::EndpointOption
20+
21+
@section dataform-env-logging Logging
22+
23+
`GOOGLE_CLOUD_CPP_ENABLE_TRACING=rpc`: turns on tracing for most gRPC
24+
calls. The library injects an additional Stub decorator that prints each gRPC
25+
request and response. Unless you have configured your own logging backend,
26+
you should also set `GOOGLE_CLOUD_CPP_ENABLE_CLOG` to produce any output on
27+
the program's console.
28+
29+
@see google::cloud::LoggingComponentsOption
30+
31+
`GOOGLE_CLOUD_CPP_TRACING_OPTIONS=...`: modifies the behavior of gRPC tracing,
32+
including whether messages will be output on multiple lines, or whether
33+
string/bytes fields will be truncated.
34+
35+
@see google::cloud::GrpcTracingOptionsOption
36+
37+
`GOOGLE_CLOUD_CPP_ENABLE_CLOG=yes`: turns on logging in the library, basically
38+
the library always "logs" but the logging infrastructure has no backend to
39+
actually print anything until the application sets a backend or they set this
40+
environment variable.
41+
42+
@see google::cloud::LogBackend
43+
@see google::cloud::LogSink
44+
45+
@section dataform-env-project Setting the Default Project
46+
47+
`GOOGLE_CLOUD_PROJECT=...`: is used in examples and integration tests to
48+
configure the GCP project. This has no effect in the library.
49+
50+
*/

‎google/cloud/dataform/doc/main.dox

Copy file name to clipboard
+47Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
/*!
2+
3+
@mainpage Dataform API C++ Client Library
4+
5+
An idiomatic C++ client library for the [Dataform API][cloud-service-docs].
6+
7+
Service to develop, version control, and operationalize SQL pipelines in BigQuery.
8+
9+
While this library is **GA**, please note that the Google Cloud C++ client libraries do **not** follow
10+
[Semantic Versioning](https://semver.org/).
11+
12+
@tableofcontents{HTML:2}
13+
14+
## Quickstart
15+
16+
The following shows the code that you'll run in the
17+
`google/cloud/dataform/quickstart/` directory,
18+
which should give you a taste of the Dataform API C++ client library API.
19+
20+
@snippet quickstart.cc all
21+
22+
## Main classes
23+
24+
<!-- inject-client-list-start -->
25+
The main class in this library is
26+
[`dataform_v1::DataformClient`](@ref google::cloud::dataform_v1::DataformClient). All RPCs are exposed
27+
as member functions of this class. Other classes provide helpers, configuration
28+
parameters, and infrastructure to mock
29+
[`dataform_v1::DataformClient`](@ref google::cloud::dataform_v1::DataformClient) when testing your
30+
application.
31+
<!-- inject-client-list-end -->
32+
33+
## More Information
34+
35+
- @ref common-error-handling - describes how the library reports errors.
36+
- @ref dataform-override-endpoint - describes how to override the default
37+
endpoint.
38+
- @ref dataform-override-authentication - describes how to change the
39+
authentication credentials used by the library.
40+
- @ref dataform-override-retry - describes how to change the default retry
41+
policies.
42+
- @ref dataform-env - describes environment variables that can configure the
43+
behavior of the library.
44+
45+
[cloud-service-docs]: https://cloud.google.com/dataform/docs
46+
47+
*/

‎google/cloud/dataform/doc/options.dox

Copy file name to clipboard
+10Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
/*!
2+
@defgroup google-cloud-dataform-options Dataform API Configuration Options
3+
4+
This library uses the same mechanism (`google::cloud::Options`) and the common
5+
[options](@ref options) as all other C++ client libraries for its configuration.
6+
Some `*Option` classes, which are only used in this library, are documented in
7+
this page.
8+
9+
@see @ref options - for an overview of client library configuration.
10+
*/

‎google/cloud/dataform/doc/override-authentication.dox

Copy file name to clipboard
+35Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
/*!
2+
@page dataform-override-authentication How to Override the Authentication Credentials
3+
4+
Unless otherwise configured, the client libraries use
5+
[Application Default Credentials] to authenticate with Google Cloud Services.
6+
While this works for most applications, in some cases you may need to override
7+
this default. You can do so by providing the
8+
[UnifiedCredentialsOption](@ref google::cloud::UnifiedCredentialsOption)
9+
The following example shows how to explicitly load a service account key file:
10+
11+
<!-- inject-service-account-snippet-start -->
12+
@snippet dataform_client_samples.cc with-service-account
13+
14+
<!-- inject-service-account-snippet-end -->
15+
16+
Keep in mind that we chose this as an example because it is relatively easy to
17+
understand. Consult the [Best practices for managing service account keys]
18+
guide for more details.
19+
20+
@see @ref guac - for more information on the factory functions to create
21+
`google::cloud::Credentials` objects.
22+
23+
[Best practices for managing service account keys]: https://cloud.google.com/iam/docs/best-practices-for-managing-service-account-keys
24+
[Application Default Credentials]: https://cloud.google.com/docs/authentication#adc
25+
26+
*/
27+
28+
// <!-- inject-authentication-pages-start -->
29+
30+
/*! @page dataform_v1::DataformClient-service-account-snippet Override dataform_v1::DataformClient Authentication Defaults
31+
32+
@snippet google/cloud/dataform/v1/samples/dataform_client_samples.cc with-service-account
33+
34+
*/
35+
// <!-- inject-authentication-pages-end -->

‎google/cloud/dataform/doc/override-endpoint.dox

Copy file name to clipboard
+25Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
/*!
2+
@page dataform-override-endpoint How to Override the Default Endpoint
3+
4+
In some cases, you may need to override the default endpoint used by the client
5+
library. Use the
6+
[EndpointOption](@ref google::cloud::EndpointOption) when initializing the
7+
client library to change this default.
8+
9+
<!-- inject-endpoint-snippet-start -->
10+
For example, this will override the default endpoint for `dataform_v1::DataformClient`:
11+
12+
@snippet dataform_client_samples.cc set-client-endpoint
13+
14+
<!-- inject-endpoint-snippet-end -->
15+
16+
*/
17+
18+
// <!-- inject-endpoint-pages-start -->
19+
20+
/*! @page dataform_v1::DataformClient-endpoint-snippet Override dataform_v1::DataformClient Endpoint Configuration
21+
22+
@snippet google/cloud/dataform/v1/samples/dataform_client_samples.cc set-client-endpoint
23+
24+
*/
25+
// <!-- inject-endpoint-pages-end -->

0 commit comments

Comments
0 (0)
Morty Proxy This is a proxified and sanitized view of the page, visit original site.