ci: Update to the CI/CD pipeline via github workflow to help cut turn… · googleapis/python-bigquery@bf58ca5 · GitHub | Latest TMZ Celebrity News & Gossip | Watch TMZ Live
Skip to content

Commit bf58ca5

Browse files
chalmerlowechalmerlowe
authored
ci: Update to the CI/CD pipeline via github workflow to help cut turn-around time (#2189)
Update to the CI/CD pipeline via github workflow to help cut turn-around time. * added github workflow * changed the number of pytest-xdist workers from "auto" to "8" (based on local tests and discussion with Tim, choosing auto sometimes takes longer to run than choosing a smaller number. I suspect this is partly because for small or short tests the overhead needed to setup a worker exceeds the time savings of having extra workers). * modified numerous tests to explicitly include a project path to avoid an attempt to find the project by making an external call via the pydata-google-auth workflow (which opens an input and waits for response from the user that never comes).
1 parent cb646ce commit bf58ca5

File tree

3 files changed

+118
-3
lines changed

3 files changed

+118
-3
lines changed

‎.github/workflows/unittest.yml

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
on:
2+
pull_request:
3+
branches:
4+
- main
5+
name: unittest
6+
jobs:
7+
unit:
8+
# Use `ubuntu-latest` runner.
9+
runs-on: ubuntu-latest
10+
strategy:
11+
matrix:
12+
python: ['3.9', '3.11', '3.12', '3.13']
13+
steps:
14+
- name: Checkout
15+
uses: actions/checkout@v4
16+
- name: Setup Python
17+
uses: actions/setup-python@v5
18+
with:
19+
python-version: ${{ matrix.python }}
20+
- name: Install nox
21+
run: |
22+
python -m pip install --upgrade setuptools pip wheel
23+
python -m pip install nox
24+
- name: Run unit tests
25+
env:
26+
COVERAGE_FILE: .coverage-${{ matrix.python }}
27+
run: |
28+
nox -s unit-${{ matrix.python }}
29+
- name: Upload coverage results
30+
uses: actions/upload-artifact@v4
31+
with:
32+
name: coverage-artifact-${{ matrix.python }}
33+
path: .coverage-${{ matrix.python }}
34+
include-hidden-files: true
35+
36+
unit_noextras:
37+
# Use `ubuntu-latest` runner.
38+
runs-on: ubuntu-latest
39+
strategy:
40+
matrix:
41+
python: ['3.9', '3.13']
42+
steps:
43+
- name: Checkout
44+
uses: actions/checkout@v4
45+
- name: Setup Python
46+
uses: actions/setup-python@v5
47+
with:
48+
python-version: ${{ matrix.python }}
49+
- name: Install nox
50+
run: |
51+
python -m pip install --upgrade setuptools pip wheel
52+
python -m pip install nox
53+
- name: Run unit_noextras tests
54+
env:
55+
COVERAGE_FILE: .coverage-unit-noextras-${{ matrix.python }}
56+
run: |
57+
nox -s unit_noextras-${{ matrix.python }}
58+
- name: Upload coverage results
59+
uses: actions/upload-artifact@v4
60+
with:
61+
name: coverage-artifact-unit-noextras-${{ matrix.python }}
62+
path: .coverage-unit-noextras-${{ matrix.python }}
63+
include-hidden-files: true
64+
65+
cover:
66+
runs-on: ubuntu-latest
67+
needs:
68+
- unit
69+
- unit_noextras
70+
steps:
71+
- name: Checkout
72+
uses: actions/checkout@v4
73+
- name: Setup Python
74+
uses: actions/setup-python@v5
75+
with:
76+
python-version: "3.9"
77+
- name: Install coverage
78+
run: |
79+
python -m pip install --upgrade setuptools pip wheel
80+
python -m pip install coverage
81+
- name: Download coverage results
82+
uses: actions/download-artifact@v4
83+
with:
84+
path: .coverage-results/
85+
- name: Report coverage results
86+
run: |
87+
find .coverage-results -type f -name '*.zip' -exec unzip {} \;
88+
coverage combine .coverage-results/**/.coverage*
89+
coverage report --show-missing --fail-under=100

‎noxfile.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -128,7 +128,7 @@ def default(session, install_extras=True):
128128
# Run py.test against the unit tests.
129129
session.run(
130130
"py.test",
131-
"-n=auto",
131+
"-n=8",
132132
"--quiet",
133133
"-W default::PendingDeprecationWarning",
134134
"--cov=google/cloud/bigquery",

‎tests/unit/test_magics.py

Lines changed: 28 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -480,6 +480,7 @@ def test_bigquery_magic_without_optional_arguments(monkeypatch):
480480
run_query_patch = mock.patch(
481481
"google.cloud.bigquery.magics.magics._run_query", autospec=True
482482
)
483+
magics.context.project = "unit-test-project"
483484
query_job_mock = mock.create_autospec(
484485
google.cloud.bigquery.job.QueryJob, instance=True
485486
)
@@ -831,6 +832,7 @@ def test_bigquery_magic_w_max_results_query_job_results_fails(monkeypatch):
831832
assert close_transports.called
832833

833834

835+
@pytest.mark.usefixtures("ipython_interactive")
834836
def test_bigquery_magic_w_table_id_invalid(monkeypatch):
835837
ip = IPython.get_ipython()
836838
monkeypatch.setattr(bigquery, "bigquery_magics", None)
@@ -861,6 +863,7 @@ def test_bigquery_magic_w_table_id_invalid(monkeypatch):
861863
assert "Traceback (most recent call last)" not in output
862864

863865

866+
@pytest.mark.usefixtures("ipython_interactive")
864867
def test_bigquery_magic_w_missing_query(monkeypatch):
865868
ip = IPython.get_ipython()
866869
monkeypatch.setattr(bigquery, "bigquery_magics", None)
@@ -1354,6 +1357,8 @@ def test_bigquery_magic_w_progress_bar_type_w_context_setter(monkeypatch):
13541357
run_query_patch = mock.patch(
13551358
"google.cloud.bigquery.magics.magics._run_query", autospec=True
13561359
)
1360+
magics.context.project = "unit-test-project"
1361+
13571362
query_job_mock = mock.create_autospec(
13581363
google.cloud.bigquery.job.QueryJob, instance=True
13591364
)
@@ -1383,6 +1388,8 @@ def test_bigquery_magic_with_progress_bar_type(monkeypatch):
13831388
run_query_patch = mock.patch(
13841389
"google.cloud.bigquery.magics.magics._run_query", autospec=True
13851390
)
1391+
magics.context.project = "unit-test-project"
1392+
13861393
with run_query_patch as run_query_mock:
13871394
ip.run_cell_magic(
13881395
"bigquery", "--progress_bar_type=tqdm_gui", "SELECT 17 as num"
@@ -1565,6 +1572,8 @@ def test_bigquery_magic_with_string_params(ipython_ns_cleanup, monkeypatch):
15651572
run_query_patch = mock.patch(
15661573
"google.cloud.bigquery.magics.magics._run_query", autospec=True
15671574
)
1575+
magics.context.project = "unit-test-project"
1576+
15681577
query_job_mock = mock.create_autospec(
15691578
google.cloud.bigquery.job.QueryJob, instance=True
15701579
)
@@ -1605,6 +1614,8 @@ def test_bigquery_magic_with_dict_params(ipython_ns_cleanup, monkeypatch):
16051614
run_query_patch = mock.patch(
16061615
"google.cloud.bigquery.magics.magics._run_query", autospec=True
16071616
)
1617+
magics.context.project = "unit-test-project"
1618+
16081619
query_job_mock = mock.create_autospec(
16091620
google.cloud.bigquery.job.QueryJob, instance=True
16101621
)
@@ -1689,6 +1700,7 @@ def test_bigquery_magic_with_option_value_incorrect(monkeypatch):
16891700
magics.context.credentials = mock.create_autospec(
16901701
google.auth.credentials.Credentials, instance=True
16911702
)
1703+
magics.context.project = "unit-test-project"
16921704

16931705
sql = "SELECT @foo AS foo"
16941706

@@ -1719,6 +1731,8 @@ def test_bigquery_magic_with_dict_params_negative_value(
17191731
run_query_patch = mock.patch(
17201732
"google.cloud.bigquery.magics.magics._run_query", autospec=True
17211733
)
1734+
magics.context.project = "unit-test-project"
1735+
17221736
query_job_mock = mock.create_autospec(
17231737
google.cloud.bigquery.job.QueryJob, instance=True
17241738
)
@@ -1760,6 +1774,8 @@ def test_bigquery_magic_with_dict_params_array_value(ipython_ns_cleanup, monkeyp
17601774
run_query_patch = mock.patch(
17611775
"google.cloud.bigquery.magics.magics._run_query", autospec=True
17621776
)
1777+
magics.context.project = "unit-test-project"
1778+
17631779
query_job_mock = mock.create_autospec(
17641780
google.cloud.bigquery.job.QueryJob, instance=True
17651781
)
@@ -1801,6 +1817,8 @@ def test_bigquery_magic_with_dict_params_tuple_value(ipython_ns_cleanup, monkeyp
18011817
run_query_patch = mock.patch(
18021818
"google.cloud.bigquery.magics.magics._run_query", autospec=True
18031819
)
1820+
magics.context.project = "unit-test-project"
1821+
18041822
query_job_mock = mock.create_autospec(
18051823
google.cloud.bigquery.job.QueryJob, instance=True
18061824
)
@@ -1852,6 +1870,7 @@ def test_bigquery_magic_valid_query_in_existing_variable(
18521870
magics.context.credentials = mock.create_autospec(
18531871
google.auth.credentials.Credentials, instance=True
18541872
)
1873+
magics.context.project = "unit-test-project"
18551874

18561875
ipython_ns_cleanup.append((ip, "custom_query"))
18571876
ipython_ns_cleanup.append((ip, "query_results_df"))
@@ -1892,6 +1911,7 @@ def test_bigquery_magic_nonexisting_query_variable(monkeypatch):
18921911
magics.context.credentials = mock.create_autospec(
18931912
google.auth.credentials.Credentials, instance=True
18941913
)
1914+
magics.context.project = "unit-test-project"
18951915

18961916
run_query_patch = mock.patch(
18971917
"google.cloud.bigquery.magics.magics._run_query", autospec=True
@@ -1917,7 +1937,7 @@ def test_bigquery_magic_empty_query_variable_name(monkeypatch):
19171937
magics.context.credentials = mock.create_autospec(
19181938
google.auth.credentials.Credentials, instance=True
19191939
)
1920-
1940+
magics.context.project = "unit-test-project"
19211941
run_query_patch = mock.patch(
19221942
"google.cloud.bigquery.magics.magics._run_query", autospec=True
19231943
)
@@ -1940,6 +1960,7 @@ def test_bigquery_magic_query_variable_non_string(ipython_ns_cleanup, monkeypatc
19401960
magics.context.credentials = mock.create_autospec(
19411961
google.auth.credentials.Credentials, instance=True
19421962
)
1963+
magics.context.project = "unit-test-project"
19431964

19441965
run_query_patch = mock.patch(
19451966
"google.cloud.bigquery.magics.magics._run_query", autospec=True
@@ -1968,9 +1989,14 @@ def test_bigquery_magic_query_variable_not_identifier(monkeypatch):
19681989
google.auth.credentials.Credentials, instance=True
19691990
)
19701991

1992+
magics.context.project = "unit-test-project"
19711993
cell_body = "$123foo" # 123foo is not valid Python identifier
19721994

1973-
with io.capture_output() as captured_io:
1995+
run_query_patch = mock.patch(
1996+
"google.cloud.bigquery.magics.magics._run_query", autospec=True
1997+
)
1998+
1999+
with run_query_patch, io.capture_output() as captured_io:
19742000
ip.run_cell_magic("bigquery", "", cell_body)
19752001

19762002
# If "$" prefixes a string that is not a Python identifier, we do not treat such

0 commit comments

Comments
 (0)

TMZ Celebrity News – Breaking Stories, Videos & Gossip

Looking for the latest TMZ celebrity news? You've come to the right place. From shocking Hollywood scandals to exclusive videos, TMZ delivers it all in real time.

Whether it’s a red carpet slip-up, a viral paparazzi moment, or a legal drama involving your favorite stars, TMZ news is always first to break the story. Stay in the loop with daily updates, insider tips, and jaw-dropping photos.

🎥 Watch TMZ Live

TMZ Live brings you daily celebrity news and interviews straight from the TMZ newsroom. Don’t miss a beat—watch now and see what’s trending in Hollywood.