Add Github Runner with Cloud Run Worker Pools to Cloud Run samples #13481
Conversation
product-auto-label
bot
added
samples
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @sagarrandive, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request adds a new sample to demonstrate how to deploy and manage self-hosted GitHub Actions runners on Google Cloud Run Worker Pools. It includes a robust autoscaling mechanism, powered by a Cloud Function, to efficiently manage runner instances based on the GitHub Actions workflow queue, thereby optimizing resource usage and cost.
Highlights
- New Sample for GitHub Runners on Cloud Run Worker Pools: This pull request introduces a complete, deployable sample demonstrating how to host self-hosted GitHub Actions runners on Google Cloud Run Worker Pools, leveraging their fast startup and shutdown capabilities for efficient resource utilization.
- Automated Scaling with Cloud Function: An autoscaling solution is implemented using a Python-based Cloud Function (
github-runner-autoscaler/main.py). This function listens to GitHubworkflow_jobwebhooks and dynamically adjusts themanualInstanceCountof the Cloud Run Worker Pool, scaling up runners when jobs are queued and scaling down when jobs complete. - Containerized Runner Deployment: The GitHub Actions runner is packaged into a Docker image (
run/github-runner/Dockerfile) based on the officialactions-runnerimage. Astart.shscript handles the configuration and execution of the runner within the container, including dynamic naming and graceful shutdown. - Comprehensive Documentation: A detailed
README.mdis included, providing step-by-step instructions for setting up the necessary GCP components, IAM roles, deploying the Cloud Run Worker Pool, configuring GitHub secrets, and deploying the autoscaler Cloud Function, along with webhook setup instructions.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request adds a sample for running self-hosted GitHub runners on Cloud Run Worker Pools with an autoscaler. Address the security vulnerability of disabled webhook signature validation, correct the cleanup logic in start.sh and the webhook setup in README.md. Remove dead code and unused dependencies in the autoscaler function.
|
|
||
| ## Configure the webhook | ||
|
|
||
| Under your repository, go to Settings -> Webhooks -> Manage webhook to configure the functions endpoint as the payload URL. Select Push events to trigger the webhook |
There was a problem hiding this comment.
The instruction to select "Push events" for the webhook is incorrect. The autoscaler function handles workflow_job events. Update the documentation to specify workflow_job events.
| Under your repository, go to Settings -> Webhooks -> Manage webhook to configure the functions endpoint as the payload URL. Select Push events to trigger the webhook | |
| Under your repository, go to Settings -> Webhooks -> Manage webhook to configure the functions endpoint as the payload URL. Select "Workflow job" events to trigger the webhook |
| # 1. Validate Webhook Signature (IMPORTANT FOR PRODUCTION) | ||
| # You need to implement this with your GitHub Webhook Secret. | ||
| # This is commented out in your original code, but critical for security. | ||
| # Example (you need to retrieve webhook_secret from Secret Manager too): | ||
| # webhook_secret = get_secret_from_secret_manager("GITHUB_WEBHOOK_SECRET") | ||
| # if not validate_signature(request, webhook_secret): | ||
| # return ("Invalid signature", 403) |
There was a problem hiding this comment.
Webhook signature validation is commented out, posing a critical security risk. Without validation, anyone can send payloads, triggering the autoscaler. Implement this validation for any real-world usage.
|
|
||
|
|
||
| # Configure the current runner instance with URL, token and name. | ||
| mkdir /home/docker/actions-runner && cd /home/docker/actions-runner |
There was a problem hiding this comment.
This command is unnecessary. The Docker container's working directory is already set to /home/docker/actions-runner. Creating a new directory is not needed and will likely cause the script to fail.
| # Initialize GitHub client | ||
| github_client = None | ||
| github_entity = None | ||
| try: | ||
| # Get GH_TOKEN from Secret Manager | ||
| client = secretmanager.SecretManagerServiceClient() | ||
| secret_name = f"projects/{PROJECT_ID}/secrets/GH_TOKEN/versions/latest" | ||
| response = client.access_secret_version(request={"name": secret_name}) | ||
| gh_token = response.payload.data.decode("UTF-8") | ||
| github_client = Github(gh_token) | ||
|
|
||
|
|
||
| if RUNNER_SCOPE == 'org': | ||
| github_entity = github_client.get_organization(GITHUB_ORG_OR_REPO) | ||
| else: | ||
| owner, repo_name = GITHUB_ORG_OR_REPO.split('/') | ||
| github_entity = github_client.get_user(owner).get_repo(repo_name) | ||
| except Exception as e: | ||
| logging.error(f"Failed to initialize GitHub client or access GH_TOKEN: {e}") |
There was a problem hiding this comment.
The block for initializing the GitHub client, along with the global variables github_client and github_entity, is not used. This is dead code that adds unnecessary complexity and dependencies. Remove this block and the corresponding unused imports.
|
|
||
| # Autoscaling parameters | ||
| MAX_RUNNERS = int(os.environ.get('MAX_RUNNERS', 5)) # Max number of concurrent runners | ||
| IDLE_TIMEOUT_MINUTES = int(os.environ.get('IDLE_TIMEOUT_MINUTES', 15)) # How long to wait before scaling down idle runners |
There was a problem hiding this comment.
The configuration variable IDLE_TIMEOUT_MINUTES is defined but not used. Remove this dead code.
| """ | ||
| auth_req, access_token = get_authenticated_request() | ||
| if not access_token: | ||
| print("Failed to retrieve Google Cloud access token. Exiting.") |
There was a problem hiding this comment.
Use the logging module instead of print for error messages for consistency and better log management.
logging.error("Failed to retrieve Google Cloud access token. Exiting.")| print(f"Successfully updated Cloud Run worker pool. Status Code: {response.status_code}") | ||
| print("Response JSON:") | ||
| print(json.dumps(response.json(), indent=2)) | ||
|
|
||
|
|
||
| except requests.exceptions.RequestException as e: | ||
| print(f"Error updating Cloud Run worker pool: {e}") | ||
| if response is not None: | ||
| print(f"Response Status Code: {response.status_code}") | ||
| print(f"Response Text: {response.text}") |
There was a problem hiding this comment.
Use the logging module instead of print for all application output to ensure logs are correctly captured, formatted, and routed in a cloud environment. Replace these print calls with logging.info and logging.error.
logging.info(f"Successfully updated Cloud Run worker pool. Status Code: {response.status_code}")
logging.info("Response JSON:")
logging.info(json.dumps(response.json(), indent=2))
except requests.exceptions.RequestException as e:
logging.error(f"Error updating Cloud Run worker pool: {e}")
if response is not None:
logging.error(f"Response Status Code: {response.status_code}")
logging.error(f"Response Text: {response.text}")| Flask | ||
| requests | ||
| google-cloud-secret-manager | ||
| google-auth | ||
| google-auth-oauthlib | ||
| google-api-python-client | ||
| PyGithub |
There was a problem hiding this comment.
This file contains unused dependencies: google-cloud-secret-manager, google-auth-oauthlib, google-api-python-client, and PyGithub. Removing them will reduce the deployment package size and improve security.
Flask
requests
google-auth
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
|
Hi @sagarrandive. This sample might be a bit too complex for this repository. I'll ping you internally on specifics. |
Add Github Runner with Cloud Run Worker Pools to Cloud Run samples. This sample also includes autoscaler for worker pool GH runner using Cloud Run Function