Migrating a Complex Drupal Website to Upsun: Architecture, Challenges, and Key Takeaways

PaaS (Platform as a Service) has transformed the way we develop and manage applications over the last decade. Thanks to them, it is possible to have scalable, easily replicable, and increasingly efficient environments.

A key player in this field is Upsun, the platform formerly known as Platform.sh. From now on, we will refer to it by its current name. We have recently worked with Upsun on managed hosting projects, and its main advantage is clear: less time invested in infrastructure means more time available for code, functionality, and quality.

We recently migrated to Upsun, a complex project with more than 30 country sites, significant traffic, and integrations with an e-commerce, a CRM (Microsoft Dynamics), and other services for automatic translations, product rating, GDPR management, etc. In this article, we want to share the considerations we had, the challenges we faced, and the lessons learned.

Architecture and sizing

Migrating to a PaaS involves choosing the plan that best suits the project. In our case, the Enterprise plan offered what we needed: multiple environments (develop, staging, and production), along with additional ones for testing new features or fixes. With the move to Upsun, these plans are now called Fixed (static resources as agreed upon in the service contract), while the Flex plans allow for dynamic scaling on demand.

Upsun isn't a typical PaaS. Instead of providing a static infrastructure, it allows you to configure your environment to your liking by setting up the services you need within the global resources you've initially contracted in Fixed mode, or you've defined in Flex mode. This makes Upsun a platform geared towards users with some experience. All the configuration for services in particular and the platform in general is made by modifying some files:

• services.yaml: With this file, we can define which services we will have in our environments, from databases to cache layers. An example would be this one, where we define a database service, a cache layer with Redis, and a Solr server to index the contents of the site and provide functionality to the search engine.

  db:
      type: mariadb:12.0
      disk: 1024
  
  cache:
      type: redis:7.2
  
  search:
      type: solr:9.6
      disk: 512
      configuration:
          cores:
            main_core:
                  conf_dir: !archive "solr/core"
          endpoints:
              solr:
                  core: main_core

• routes.yaml: Responsible for the application's access routes, here we can set our base URL, redirects, URLs for other apps on our project, etc. In this example, we define the basic project URL based on our site URL configured on the Upsun dashboard. As we had to add more rules to our migrated project, we had to add this file, but if your project only uses the default URL, it is not needed, as Upsun provides a default configuration:

  routes:
    "https://{default}/":
      type: upstream
      upstream: myapp:http

• .platform.app.yaml: This is the central file for defining the PHP version, site name, environment variables, dependencies, or mount points.

  name: 'my site'
  
  # The runtime the application uses.
  type: 'php:8.3'
  
  # Packages in web container
  dependencies:
      php:
          composer/composer: '^2.8.2'
      nodejs:
          n: "*"
  # Environment variables
  variables:
      env:
          N_PREFIX: /app/.global
  # Relations with the services defined in services.yaml
  relationships:
      database: 'db:mysql'
      redis: 'cache:redis'
      solr: 'search:solr'
      
  # The hooks allow us to run differents scripts on build, deploy, etc.
  hooks:
      build: echo "my build hook"
      deploy: echo "Deployin app"
  
  # Define crons processes
  crons: 
      drupal:
          spec: '*/5 * * * *'
          commands:
              start: 'cd web ; drush cron'
              
  # Allow to deep configure the web server rules
  web: ...

Deployments and orchestration

Upsun uses isolated environments based on LXC and namespaces, with its own orchestration system inspired by Docker concepts. Each service (PHP, database, Redis, etc.) runs in its own container, automatically managed by the platform.

Each deployment completely rebuilds the environment, ensuring it is always clean and consistent. Additionally, the platform offers hooks at different stages of the process for tasks such as artifact generation, configuration import, or cache clearing, all configurable in the .platform.app.yaml file.

A common practice on PaaS is to link deployments to a code repository, with different environments tied to specific branches. When it comes to managing code and deployments on Upsun, there are two primary approaches:

• Single Repository: The entire project resides in a single repository linked directly to the infrastructure. The build artifact is generated during the deployment process itself, using the hooks available on Upsun.

  

• Separated Repositories: This approach uses two repositories: one independent of Upsun for development and a second one linked to Upsun where the build artifacts are uploaded from a CI/CD pipeline.

  

While here at Metadrop, we prefer the second approach, as it allows for a more thorough control over what gets deployed to our environments (including functional tests, visual regression, etc.), it's not always possible to use a second repository, or the goal may be to keep the entire process contained within the Upsun platform. Both approaches are perfectly valid.

In our case, we integrated our own artifact-building tool into Upsun's deployment workflow, and the integration was quick and straightforward. This allowed us to maintain a Continuous Integration/Continuous Deployment (CI/CD) process aligned with our internal practices, without compromising the environment's stability.

Preparing for the migration

Days before the switch, we prepared the environments in Upsun:

• Organization and accounts: Support service created the organization, and we managed users and permissions from the Upsun dashboard, giving access to developers and administrators
• Repository: While it is possible to use Upsun's integrated repository for our site's codebase, we opted to continue using our existing repository and activated Upsun's integration with that source. This approach allowed us to retain our established incident management, CI, and Wiki systems, and also prevented the need for a full code and history migration.

• Database synchronization: Database synchronisation will only be performed in certain environments, as we will discuss later. Generally, it is sufficient to perform a dump of the old environment and import it into Upsun. This can usually be accomplished by connecting to the server via SSH or by using Upsun's provided CLI tool. Note: Extreme care must always be taken with this data. Once imported, the dumps must be immediately deleted.

  platform sql -p <PROJECT_ID> -e <ENVIRONMENT_NAME> < live_dump.sql

• Static files synchronization: This task might seem straightforward at first glance, as it primarily involves moving files from one server to another. However, when the volume of information to be moved is in the order of tens of gigabytes, it presents a significant challenge. This is one of the most problematic parts of our migration. Beyond the sheer volume of data, the added complexity of this task is significantly determined by the possibility of establishing a direct connection from the original server to Upsun to initiate synchronisation. If such a direct server connection is feasible, a simple rsync synchronisation will be enough:

  # Example rsync command
  rsync -avz /mysite/web/sites/default/files/ user@upsun_server:/path/to/destination/web/sites/default/files/

  If a direct connection is not possible, it will be necessary to synchronise data via a third device or attempt to initiate rsync through an SSH tunnel, always ensuring data security.

  • Initial installation of environments: Once the codebase was fully integrated into the platform, all static files and databases were synced, and the environments were active, we proceeded to prepare the necessary artifact generation and deployment scripts for the various environments. On this occasion, we were compelled to use the single repository approach, meaning the artifact generation occurred as part of the deployment process itself. With these scripts, we deployed the following three production environments:
    • Dev: Quick deployment, with a Drupal installation from configuration, and default content created with the default content module.
    • Staging: This is an environment with a real database from production. As we have an existing database, we don't run any Drupal installation; we run the usual Drupal deployment steps after the artifact building instead: make a database backup and run "drush cache:rebuild, drush updatedb, drush config-import, and drush cache:rebuild again.
    • Live: Same as staging, but for the first time, we get a backup from the previous infrastructure.

With all environments fully prepared, we execute our comprehensive suite of tests. This typically covers the majority of the site and all critical functionalities, serving as a safeguard to maintain the stability of our sites

Furthermore, once the main environments are established, the Upsun dashboard allows us to configure whether any new branch should trigger the creation of a new environment, and from which parent environment it should inherit, in our case, 'develop'. There is also the option to generate environments solely upon the creation of Merge/Pull requests.

When a new environment inherits from another, it signifies that upon deployment, it will receive a copy of the parent environment's database. While not strictly essential for our 'develop' environment, which is installed from configuration, this feature proves particularly useful when generating environments that inherit from 'staging' or 'production'.

To minimize disruption, we also reduced the DNS TTL, so that propagation would be as fast as possible when the migration day arrives.

Executing the migration

The transition day is always critical. To ensure success, we followed a very structured procedure:

1. Freeze changes in the old production environment to avoid inconsistencies: For a straightforward site without registered user interaction, this can be achieved simply by requesting that editors not access the site. On more complex sites, this can be accomplished by setting the database to read-only mode or by using modules such as Read-only mode.
2. Full system backup and syncronization: Database must be imported from the old server to Upsun again, along with the static files (images, CSS, etc.). Remember that the database can be synchronized with Upsun CLI. If static files can be synchronized with rsync, it will be enough with an rsync cumulative update; if not, probably a full synchronization will be required.
3. Run all automated and manual tests: Once everything is synchronised, in our case, we then run the test suite specifically built for each project. This suite typically covers the majority of the site and all critical functions, but it does not negate the need for an additional manual verification before proceeding to the final step.
4. DNS/CDN change: once we confirmed that everything was working as expected, the site is ready to be published. At this stage, the DNS must be updated send all traffic to Upsun servers. We must also consider any intermediate layers, such as Fastly or other CDNs.

Thanks to this sequence, the downtime was minimal, and the migration was perceived as practically seamless for end-users. 

Considerations if you are migrating your sites to Upsun

Beyond this specific case, there are several key aspects to consider before taking the leap:

• Sizing and budget: Carefully evaluate the plan you need and the associated cost. Choosing between Fixed or Flex can make a difference in scalability and resource optimization.
• Deployment strategy: Although you can directly link your development repository to Upsun, our key recommendation is that the repository connected to the platform should be the already compiled artifact, using, for example, drupal-artifact-builder within a pipeline, using the previously mentioned "Separated repositories" strategy. This way, you have better control over what and when is deployed, keeping a separate repository for development.
• Available services and versions: Check that Upsun offers all the services your infrastructure needs and in the correct version in their list of services. If not, do not hesitate to ask their team if it's possible to add them.
• Not a typical PaaS: If you are seeking a conventional PaaS where everything is pre-built and you merely upload your data, then Upsun might not be the ideal solution for you. Upsun offers greater flexibility, but at the cost of increased complexity. In our specific case, we consider this a positive trade-off; however, the journey is not always straightforward. On occasion, throughout these migrations, we have encountered challenging hurdles that required significant effort to overcome

Conclusions

Upsun is a PaaS that offers greater infrastructure flexibility, which consequently introduces a degree of complexity during the setup phase. For us, this is a significant advantage, particularly valuable in more complex projects where flexibility is crucial.

Ultimately, migrating your project to Upsun can be a complex process. At Metadrop, we are official Upsun partners. If you have any questions or need assistance with the migration, please don’t hesitate to get in touch with us.