Documentation update - devel #615
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
…cs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server.
* Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call.
…104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code.
* Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump.
* Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no.
* Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last.
* Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials.
* Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once.
* Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role.
* Fixing GRANT query for MySQL > 8.0. * Create what though?
…f static database users are being used. (#133)
* Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks.
* Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs.
* Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work.
…159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role.
* Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp.
* Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var.
* Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once.
* Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role.
* Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild.
* Updating-cron-path-fixing-mailto * Adding-md * Using-cronvar-module * Fixing-linting-condition * Adding-mailto-dynamically-for-specific-jobs * Fixing-incorrect-crontab-entries * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-condition-for-crons * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Updating-cron-path-fixing-mailto * Mailto-filtering * Polishing-cron-output * Fixing-syntax * Filtering-test-crons-mailto * Testing-trim-jinja * Cronjob-path-fix * Reverting
* Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Jinja2 template looking for settings.php using wrong path. * Fixing config_generate for Drupal. * Making host checking more robust. * Removing variable declaration that will break host check. * Allow us to use deploy_code with a completely custom build and do nothing. * Make PHP cachetool install optional. * Fixing bug introduced by failed_when - need to check for return code on mount points. * Moving where opcache clears get called so they can be excluded if necessary. * Fixing linting errors around white space. * The unarchive module requires 'dest' to be a directory. * Fixing Drupal example templates. * Logically splitting tests and removing LGD because it's broken. * Adding extra plugin LGD needs. * Clearing stat and apcu cache for PHP by default. * Adding some perms handling for Drupal 8+ config sync where deploy and web user are different. * Rolling back APCu cache by default as the package must be installed. * Making the remount squashfs code more defensive by using a block. * Trying with 'throttle' keyword to restrict to one machine at a time. * Sadly 'throttle' seems to work on a task level only, not a block level. * Adding hash salt generation for Drupal 8+. * Trying a different syntax for passing params to the password plugin. * Forgot the constants import in plugin. * Reinstating the SOPS plugin - we *do* need it. * Making online/offline tasks use current live code/database. * We need build paths to bring the site back online if it's a squashfs build. * Ensuring _common.sh includes /etc/profile.d for Python venv path. * Making linter happy. * Using the drush_live_bin var in sync jobs for Drupal 8+. * Fixing database dump revert. * Fixing gunzip command. * Having to do a double step to unpack SQL file in /tmp. * Load bash profiles before executing a build. * Fixing linting error, use -n instead of ! -z. * Adding support for specifying path to Ansible and path to Python interpreter. * Moving installation of cachetool to application includes, where they are PHP apps. * Fixing minor typo. * Cleaning up namespacing. * Making quotes consistent. * Making quotes consistent. * Making quotes consistent. * Allowing for Drupal cron rollback on revert. * Only running maintenance mode enable on deploy. * Moving previous_deploy_path out of the squashfs block. * Committed installer by accident. * Trying to fix CI. * Ensuring ASG handling only runs once per server. * Fixing namespaces flagged by linting. * Only need to look up the SquashFS image once per cluster.
* Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location.
* Fixing Mautic deployment workflow. * Ensuring symlink destinations exist. * The docroot folder doesn't exist yet, so ensure it gets created. * Mautic cache role obsolete as it now uses Symfony properly. * Updating Mautic console bin location. * Fixing missing code.
* Fixing docs publishing. * Updating docs.
* Fixing docs publishing. * Updating docs. * Updating name of job.
* Fixing docs publishing. * Updating docs. * Updating name of job. * Making docs publishing changes from ce-provision in ce-deploy.
* Fixing docs publishing. * Updating docs. * Updating name of job. * Making docs publishing changes from ce-provision in ce-deploy. * Making devel PR for docs and better error handling.
Co-authored-by: Code Enigma CI <sysadm@codeenigma.com>
* Merging 1.x down into devel (#269) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Target host checking pr devel (#270) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Sync enhancements pr devel (#272) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Bug fixes pr devel (#274) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role. (#155) * Make config imports during syncs optional (#157) * Use a different variable for config imports during a sync so they can be optional. * Actually, no. Use a variable to determine if config should be imported during a sync. * Fix some logic with config import during syncs and add a comment to explain when the task runs. * Squashfs pr 1.x (#153) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Adding a revert behaviour for SquashFS. * Making some wording a little less ambiguous. * Fixing bad image path and adding clauses to stat check. * Trying to make remounting SquashFS images a bit safer. * Adding when clauses to SquashFS image path check. * Removing revert code, as it cannot work. * Add cache clears to Drupal deployments, before DB updates and stuff (#159) * Add cache clears to Drupal deployments, before database updates and config import are run. * Move cache clears out of meta roles and only run in database_apply role on non-initial builds. * Add cache clear stuff to drupal7 database_apply role. * Avoid leaving exponentially growing sqsh files in build locations! (#164) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Exclude sqsh file pr 1.x (#167) * Avoid leaving exponentially growing sqsh files in build locations! * Use mount point instead of /tmp for stowing previous sqsh file. * Making sure image/tarball filenames are build specific in /tmp. * Removing unnecessary lines in Drupal config generation. (#169) * Removing unnecessary lines in Drupal config generation. * Drupal 7 lacked the install_command var. * Ensuring dump directory exists on backup step. (#172) * Ensuring dump directory exists on backup step. * Acts on shared storage, so can run once. * Allowing Drupal 7 jobs to disable cron. (#174) * Suppress db revert pr 1.x (#177) * Supporting option to suppress reverting backups. * Fixing namespacing in database_backup role. * Fixing bad assumption that databases will have TCP connections. (#179) * Handling the 'drush deploy' command more elegantly for Drupal 8+. (#180) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Attempt to clear the opcache during Drupal deployments. (#182) Co-authored-by: Emlyn Kinzett <emlyn.kinzett@codeenigma.com> * Better drush deploy support pr 1.x (#185) * Handling the 'drush deploy' command more elegantly for Drupal 8+. * Changing logic to AND for cache rebuild. * Em's clear cache code needs to run for each 'site' too. * Cron job schedule params pr 1.x (#190) * cron job schedule params, namespaces * drupal7-cron-params * cron-job-schedule-params-fix-defaults * Adding option to stop services that might interfere with a squashfs mount. (#193) * Drush refactor pr 1.x (#197) * Updating docs. * Consistent drush handling in cron. * Linting cruft and removing unimplemented drush.yml handling from D7. * Refactor drush role to support new GitHub release format. * Updating drush README. * Casting the first part of the drush version string as an integer. * Refactoring integer handling. * Major version string will always be a string, assuming below 8 will never exist. * Downloaded drush archive is remote, not local. * Refactoring handling of .bin directory. * Supporting drush-launcher. * Removing drush installation support for Drupal 8 and above. * Bad variable name in Drupal cron jobs. * Fixing maintenance mode command linting and consistency. * Moving chdir to args to see if it helps. * Trying an actual 'cd' command inline. * Cannot '&&' in a cmd. * Trying drush's --root flag to set Drupal path. * With --root we do not need chdir for drush. * Revert "With --root we do not need chdir for drush." This reverts commit d07aaf798a8f5699ed669f9411db2810e2bcd9d8. * Better deploy_code role docs. (#213) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Cachetool settings and drupal7 tasks fix pr 1.x (#211) * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool_settings_and_drupal7_tasks_fix * cachetool settings fixing condition and namespace * fixing cachetool adapter options * fixed var * fixed var * fixed var * previous_cachetool_remove_before_install_new (#224) * previous_cachetool_remove_before_install_new * namespace fix * Fix cachetool removal pr 1.x (#227) * fix_cachetool_removal_step * fix task name * Fix cachetool removal pr 1.x (#229) * fix_cachetool_removal_step * fix task name * fix_cachetool_removal_task * fix_cachetool_removal_task * Documentation enhancements pr 1.x (#218) * Better deploy_code role docs. * roles path error in docs. * roles path error in docs. * Adding a note about deploy_previous handling for squashfs. * Reference incorrect role for deploy user sudo perms. * Minor edits to frontpage README. * Rebuilt docs. * Accidentally overwrote docs change. * Drush refactor pr 1.x (#231) * Fixing sync jobs. * Fixing bad task title. * Drush refactor pr 1.x (#234) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Drush refactor pr 1.x (#236) * Fixing sync jobs. * Fixing bad task title. * Trying to fix D7 drush issues with chdir. * Moving D7 drush commands back to 'shell'. * Change php detection for fastcgi and cachetool pr 1.x (#238) * change_php_detection_for_fastcgi_and_cachetool * fix_typo * change_php_detection_for_fastcgi_and_cachetool_no_sudo * Cachetool bin per site pr 1.x (#243) * cachetool_bin_per_project * cachetool_bin_per_site_fix * cachetool_bin_per_site_fix2 * apcu no longer has an 'all' option. (#245) * Refactoring cachetool path setting to match other squashfs vars. (#247) * Cachetool pr 1.x (#249) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Ensuring we can prevent features and ctools deployments in Drupal 7. (#240) * Cachetool pr 1.x (#251) * Refactoring cachetool path setting to match other squashfs vars. * Refactoring cachetool_bin handling. * Forgot to remove the default() filter from squashfs var setting. * Refactoring clauses slightly to simplify. * improving_old_builds_cleanup (#254) * improving_old_builds_cleanup * improving_old_builds_cleanup_fix * Adding escaped backticks to db names to be safe. (#256) * Mysql db name pr 1.x (#258) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Mysql db name pr 1.x (#260) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Mysql db name pr 1.x (#262) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Mysql db name pr 1.x (#264) * Adding escaped backticks to db names to be safe. * Looks like Ansible auto-escapes backticks. * Adding backticks to database names for creating MySQL users too. * Adding extra mysqldump flags to stop restores failing on RDS. * Removing ignore_errors. * Updating mysqldump query. * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Bug fixes pr 1.x (#266) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Bug fixes pr 1.x (#268) * Fixing linting so CI can work. * More linting fixes. * Downgrading Ubuntu due to Docker issues with ce-dev and latest. * Making pipefail shell commands use /bin/bash. * Adding pipefail code to cachetool installer block. * Adding executable for drush install checker. * Fixing D7 ctools behaviour to match features handling. * Making sync paths build unique. * Adding --host parameter to build.sh for pre-deploy host checking with Ansible. (#271) * Making syncs safer and more efficient. (#273) * Jinja2 template looking for settings.php using wrong path. --------- Co-authored-by: EmlynK <emlyn.kinzett@codeenigma.com> Co-authored-by: Dionisio <dionisiofernandez83@gmail.com> Co-authored-by: tymofiisobchenko <104431720+tymofiisobchenko@users.noreply.github.com> * Revert "Bug fixes pr devel (#274)" (#276) This reverts commit 3e2977500ee9fb68c8e5b01e836e07718549ab66. * Bug fixes pr devel (#277) * Defer nightly backups, disable ASG processes during syncs and run syncs with backup (#94) * Change location of nightly backup script and delegate the cron that runs it to the deploy server, if required (for ASGs). * Set up nightly backup crons in separate files in /etc/cron.d * Can't put site cron files in /etc/cron.d because the deploy user doesn't have perms. * Try and add the ability to sync a site using a nightly backup instead of a fresh DB dump. * Use Ansible modules to look up RDS host and to copy the nightly backup into place. * Delegate PATH setup in db backup cron to localhost. * Shell bad. Command good. But makes it convoluted. Oh well. * Used wrong database name in source database copy. * Try and disable the ReplaceUnhealthy auto scale process during syncs. * Fix Drupal cron roles when deferring to deploy server. * Added deploy.yml examples for Drupal 9 and Localgov. Updated tests (#97) * Adding a new SimpleSAMLphp meta role. (#100) * Allowing users to set cachetool version properly. (#102) * Deploy ami pr 1.x (#106) * Including an ami.yml for packing AMIs on build finale. * New api_call role, focused on GitLab for now. * Optionally trigger an infra build with an API call. * Making the MySQL dump command for routine back-ups less aggressive. (#104) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Fix database backups pr 1.x (#109) * Making the MySQL dump command for routine back-ups less aggressive. * Making max_allowed_packets a variable we can set. * Adding dump command to a re-usable central var. * Renaming _init var so it's easier to distinguish in the code. * Defaults file must be the first param for mysqldump. * Fix MySQL backup deferral. (#110) * Files recurse fix pr 1.x (#112) * Don't recurse through site directory when setting permissions during config_generate step. * Update drupal7 config_generate perms update task and use true/false instead of yes/no. * Improve multisite support (#115) * Pass -l option in drush commands to specify site name, which should match the site folder name. * Move -l option in drush commands to the first option instead of the last. * Static credentials handling fix pr 1.x (#119) * Attempt to fix static credentials handling when deploying to an ASG. * Include build_type in static password file location. * Move to using new location for static credentials handling, but try to catch any legacy stuff too, for now. * Add a couple of comments to legacy handling of static credentials. * Making contents of deploy tar 'ownerless'. (#117) * Implement file syncing (#124) * Add a files_sync role. * Clear up some comments, add temp_dir variable and use rsync with command module instead of synchronize module, in files_sync role. * Need trailing slash on src when syncing files to destination server so the contents get synced and not the parent directory itself. * Use rsync instead of copy when syncing files to target server as copy is so slow. * Ensure file sync tasks are run only once. * Create Drupal-specific sync roles (#128) * Add a drupal_sync_tasks role to run Drupal specific tasks during syncs. * Move sync database_apply files into tasks subdirectory. * Woops, used bad role names in sync database update roles. * Remove feature reverting from drupal7 database_apply sync role. * Fixing GRANT query for MySQL > 8.0. (#131) * Fixing GRANT query for MySQL > 8.0. * Create what though? * Use IF NOT EXISTS when creating database user as that command fails if static database users are being used. (#133) * Attempt to fix syncs whenever the 'dump' type is used for source or target. (#149) * Squashfs pr 1.x (#150) * Adding SquashFS option to syncing. * Make 'tarball' the default mount type so nothing existing breaks. * Altering deploy_path so we can build in another location for SquashFS builds. * Ensuring the build dir exists if doing a SquashFS build. * Force symlink creation as the deploy dir may not exist yet. * Running mount commands with sudo. * Tweaking link command and destination for live links. * Slight bug in link path handling for SquashFS. * Checking for existing mount and using remount operation. * Stop ce-deploy trying to delete from read-only SquashFS mount. * Formatting error, these special jinja2 things are not filters - no spaces. * Reloading services to make sure mounting doesn't fail. * Picking more sensible loop var name. * Moving to shell for mount check and fixing jinja2 filter names. * Trying with the posix mount module instead of command. * Working through user/sudo issues. * Altering symlink handling slightly so we have the deploy directory always set. * Simplifying 'when' checks. * Check deploy_code.mount_type is defined when setting facts in init role.…
* Fixing docs publishing. * Updating docs. * Updating name of job. * Making docs publishing changes from ce-provision in ce-deploy. * Making devel PR for docs and better error handling. * Making job skipping work properly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Automated pull request created by GitHub Actions because of a documentation update.