From 50919e26f5b64903857232ff5316df9c63b92ce9 Mon Sep 17 00:00:00 2001 From: Kevin Strider Date: Thu, 26 May 2022 04:24:33 -0700 Subject: [PATCH] public-releases.mdx (Under the Hood - Public Flipper Releases) Summary: Restyle of page, including changes to spelling, grammar, links, and structure (where relevant). Reviewed By: passy Differential Revision: D36666339 fbshipit-source-id: 4c57746bd62fd52fa8dff4fda3098c443941ae30 --- docs/extending/public-releases.mdx | 152 +++++++++++------------------ 1 file changed, 59 insertions(+), 93 deletions(-) diff --git a/docs/extending/public-releases.mdx b/docs/extending/public-releases.mdx index cd1bc6ef3..cfe031b25 100644 --- a/docs/extending/public-releases.mdx +++ b/docs/extending/public-releases.mdx @@ -4,161 +4,127 @@ title: Public Flipper Releases --- import useBaseUrl from '@docusaurus/useBaseUrl'; -This document outlines how Flipper releases work on GitHub. +This page outlines how Flipper releases work on GitHub. ## Kick-Off A release is kicked off by a special commit that has a subject with the format -`Flipper Release vX.Y.Z`, e.g. [4fa2c9761](https://github.com/facebook/flipper/commit/4fa2c9761a8359c65ccc62fee76490572616f0c1). +`Flipper Release vX.Y.Z` (see the GitHub [example](https://github.com/facebook/flipper/commit/4fa2c9761a8359c65ccc62fee76490572616f0c1)). -This is triggered from a bot within Facebook that runs [prepare-release.sh](https://github.com/facebook/flipper/blob/main/scripts/prepare-release.sh) -but the only thing special about the commit is its subject line. Anyone could run the -script and would kick off the remaining jobs once the commit lands in main. +This is triggered from a bot within Facebook that runs [prepare-release.sh](https://github.com/facebook/flipper/blob/main/scripts/prepare-release.sh). The only thing special about the commit is its subject line. Anyone could run the script and kick off the remaining jobs once the commit lands in main. -The commit bumps the version of Flipper Desktop as well as various SDK components -and libraries that are to be published to npm and other package repositories. +The commit bumps the version of Flipper Desktop as well as various SDK components and libraries that are to be published to npm and other package repositories. -Importantly, it is immediately followed by a "SNAPSHOT" commit (e.g. [02a56da3f](https://github.com/facebook/flipper/commit/02a56da3f5440b35616604f73167307319cca48f)) which sets -the version of our Java dependencies to `X.Y.(Z+1)-SNAPSHOT` (that's the -patch version incremented by one and a `-SNAPSHOT` suffix appended). -That's a weird Maven-ism which allows us to continuously publish snapshot -releases from the main branch. +Importantly, it is immediately followed by a 'SNAPSHOT' commit (see the GitHub [example](https://github.com/facebook/flipper/commit/02a56da3f5440b35616604f73167307319cca48f)), which sets the version of our Java dependencies to `X.Y.(Z+1)-SNAPSHOT` (that's the patch version incremented by one and a `-SNAPSHOT` suffix appended). It's a weird Maven-ism that enables us to continuously publish snapshot releases from the main branch. ## Desktop Release The release process for the desktop app is entirely driven by [GitHub Actions](https://github.com/features/actions). -The entry point is the [release.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/release.yml) workflow -which is triggered by changes to the `desktop/package.json` file on the main branch. +The entry point is the [release.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/release.yml) workflow, which is triggered by changes to the `desktop/package.json` file on the main branch. -GitHub Actions has a limitation we need to work around here: It allows push events to as triggers for -a workflow, but not individual commits. This requires us to first scan through all potential commits -of a push to identify commits that match the aforementioned versioning schema. -This is done through a [custom action](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.github/workflows/release.yml#L15-L18). +GitHub Actions has a limitation that we need to work around: it enables push events to act as triggers for a workflow, but not individual commits. This requires us to first scan through all potential commits of a push to identify commits that match the aforementioned versioning schema. This is done through a [custom action](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.github/workflows/release.yml#L15-L18). -Only if this action is successful, the remaining steps are taken which +If this action is successful, the remaining steps are taken: -- Check out the versioning commit. -- Create a git tag for it. -- Create a GitHub release for the tag. -- Build releases for Linux, Mac and Windows. -- Upload all these releases to temporary storage. -- Download the release artifacts and attach them to the GitHub release. -- Dispatch to separate workflows for publishing Cocoapods and npm packages (see below). +* Check out the versioning commit. +* Create a git tag for it. +* Create a GitHub release for the tag. +* Build releases for Linux, Mac and Windows. +* Upload all these releases to temporary storage. +* Download the release artifacts and attach them to the GitHub release. +* Dispatch to separate workflows for publishing Cocoapods and npm packages (see below). ## iOS Release -iOS releases are run in GitHub Actions but exist as a separate workflow. They can be triggered in three ways: +iOS releases are run in [GitHub Actions](https://github.com/features/actions) but exist as a separate workflow. They can be triggered in three ways: 1. When a tag is pushed. -2. By manually triggering the workflow (see below). -3. Through a `dispatch_workflow` event which is issued as a last step of the desktop release process. +2. By manually triggering the workflow (see the following screenshot). +3. Through a `dispatch_workflow` event that is issued as a last step of the desktop release process. Manually triggering a workflow -[The workflow](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-pods.yml) follows the default Cocoapods update procedure, bumps and published both the Flipper and FlipperKit pod -and finally creates a [pull request](https://github.com/facebook/flipper/pull/1754) containing the updated references. This PR must be manually merged. +The [workflow](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-pods.yml) follows the default Cocoapods update procedure, bumps and publishes both the Flipper and FlipperKit pod, then creates a [Pull Request](https://github.com/facebook/flipper/pull/1754) (PR) containing the updated references. This PR must be manually merged. +:::note **Authentication** is managed through the secret environment variable `COCOAPODS_TRUNK_TOKEN`. +::: ## npm releases The JavaScript libraries published as part of a Flipper release closely follow the iOS release procedure. -Same as before, it is a workflow that is triggered by one of the three events, which should in all but -exceptional circumstances be a dispatch event. The workflow is defined in [publish-npm.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-npm.yml). +As with the [iOS release](#ios-release), it's a workflow that is triggered by one of the three events, which should, in all but exceptional circumstances, be a dispatch event. +The workflow is defined in [publish-npm.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-npm.yml). -From there, we use a [script](https://github.com/facebook/flipper/blob/main/desktop/scripts/bump-versions.ts) to -bump the versions of our Yarn workspaces, and publish all public packages (`flipper`, `flipper-babel-transformer`, ...) -and our React Native bindings. +From there, we use a [script](https://github.com/facebook/flipper/blob/main/desktop/scripts/bump-versions.ts) to bump the versions of our Yarn workspaces, and publish all public packages (`flipper`, `flipper-babel-transformer`, ...) and our React Native bindings. +:::note The **authentication** to npm is managed by a secret environment variable called `FLIPPER_NPM_TOKEN`. +::: ## Android Release -Android has three types of jobs currently running. The `snapshot` job is an -outlier in that it still runs on CircleCI. This gives us some additional -capacity as these jobs can take quite a while and the occasional failure -due to timeouts or network errors isn't a dealbreaker. +Android has three types of jobs currently running: -The three jobs are: +1. [snapshot](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) - runs on every commit on the main branch and publishes 'SNAPSHOT' releases to Maven Central. It runs on CircleCI. +2. [publish-android](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-android.yml) - usually triggered by a `dispatch_workflow` event. It uploads our Java artifacts to Maven Central and attaches the Android sample app to the release page on GitHub. It runs on GitHub Actions. +3. [android-sample](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) - runs on every push and open pull request. It builds the sample and tutorial apps and uploads the sample APK as artifact for easy debugging and testing. -1. The [`snapshot`](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) job runs on every commit on the main branch and publishes "SNAPSHOT" releases to Maven Central. It runs on CircleCI. -2. The [`publish-android`](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-android.yml) job is usually triggered by a `dispatch_workflow` event. It uploads our Java artifacts to Maven Central and attaches the Android sample app to the release page on GitHub. It runs on GitHub Actions. -3. The [`android-sample`](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) job runs on every push and open pull request. It builds the sample and tutorial apps and uploads the sample APK as artifact for easy debugging and testing. +The `snapshot` job is an outlier in that it still runs on CircleCI. This gives us some additional capacity as these jobs can take quite a while and the occasional failure due to timeouts or network errors isn't a dealbreaker. ### CircleCI Configuration -The Android snapshot build is run on -[CircleCI](https://app.circleci.com/pipelines/github/facebook/flipper?branch=main) -and configured in -[`.circleci/config.yml`](https://github.com/facebook/flipper/blob/main/.circleci/config.yml) +The Android snapshot build is run on [CircleCI](https://app.circleci.com/pipelines/github/facebook/flipper?branch=main) and configured in [.circleci/config.yml](https://github.com/facebook/flipper/blob/main/.circleci/config.yml). There are two potential points for breakage: -1. The [base image](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.circleci/config.yml#L5) used in the build instructions refers to a specific SDK version and requires occasional updating. -2. The [platform installation](https://github.com/facebook/flipper/blob/b5e613141e98528f45d3d2864e08278b1c7d4973/.circleci/config.yml#L23) through the `sdkmanager` tool of the Android SDK may require additional SDKs or NDKs to be installed if they're not part of the base image. +1. [Base image](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.circleci/config.yml#L5) - used in the build instructions refers to a specific SDK version and requires occasional updating. +2. [Platform installation](https://github.com/facebook/flipper/blob/b5e613141e98528f45d3d2864e08278b1c7d4973/.circleci/config.yml#L23) - runs through the `sdkmanager` tool of the Android SDK. It may require additional SDKs or NDKs to be installed if they're not part of the base image. -One non-obvious aspect is that of **authentication** for uploads. The repository contains a symmetrically encrypted -copy of our credentials to Sonatype (for Maven Central). The [snapshot release script](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/scripts/publish-android-snapshot.sh) -decodes the file on the fly by using a secret Circle CI exposes through an environment variable. +One non-obvious aspect is that of **authentication** for uploads. The repository contains a symmetrically encrypted copy of our credentials to Sonatype (for Maven Central). The [snapshot release script](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/scripts/publish-android-snapshot.sh) decodes the file on the fly by using a secret Circle CI exposes through an environment variable. ### GitHub Action Workflow -As with the iOS release before, the workflow for Android releases is triggered by three types of events: +As with the [iOS release](#ios-release) (see above), the workflow for Android releases is triggered by three types of events: 1. When a tag is pushed. 2. By manually triggering the workflow. -3. Through a `dispatch_workflow` event which is issued as a last step of the desktop release process. +3. Through a `dispatch_workflow` event, which is issued as a last step of the desktop release process. -In normal circumstances, the third event will kick off an Android release build. -The workflow is defined in [`publish-android.yml`](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/.github/workflows/publish-android.yml). +In normal circumstances, the third event will kick off an Android release build. The workflow is defined in [publish-android.yml](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/.github/workflows/publish-android.yml) in GitHub. -We first install two NDK versions that are required by our dependencies. To publish release artifacts -(i.e. non-`SNAPSHOT` artifacts), Maven Central requires them to be signed with a GnuPG key. The -only requirement about the key itself is, that it needs to be exported to a Keyserver. Ours -is published [to the Ubuntu Keyserver](https://keyserver.ubuntu.com/pks/lookup?search=Flipper+Bot+%28I+sign+Flipper+releases%29+%3Crealpassy%40fb.com%3E&fingerprint=on&op=index). +We first install two NDK versions that are required by our dependencies. To publish release artifacts (non-`SNAPSHOT` artifacts), that Maven Central, requires them to be signed with a GnuPG key. The +only requirement about the key is that it needs to be exported to a Keyserver. Ours is published to the [Ubuntu Keyserver](https://keyserver.ubuntu.com/pks/lookup?search=Flipper+Bot+%28I+sign+Flipper+releases%29+%3Crealpassy%40fb.com%3E&fingerprint=on&op=index). -To publish your own key, run +To publish your own key, run the following: ```bash gpg --send-keys --keyserver keyserver.ubuntu.com ``` -For the initial setup, the secret keyring was exported as `gpg2 --export-secret-keys | base64` and stored -as secret on GitHub with the name `GPG_KEY_CONTENTS`. As part of the workflow, it is written to disk after reversing the base64 encoding. -The key id and key password are subsequently stored in the `gradle.properties` along with the path to the key. Paths here need to be -absolute, otherwise Gradle will look them up relative to the sub-projects (`android/`, `android/sample`, ...). +For the initial setup, the secret keyring was exported as `gpg2 --export-secret-keys | base64` and stored as a secret on GitHub with the name `GPG_KEY_CONTENTS`. -Maven Central is managed by Sonatype. To sign up follow [their guide](https://central.sonatype.org/pages/ossrh-guide.html) which -involves creating a JIRA account and opening an issue to apply for the `com.facebook` namespace. You will need -to find an existing member of this namespace to vouch for you. While this is a lot, it ensures that nobody +As part of the workflow, it is written to disk after reversing the base64 encoding. The key id and key password are subsequently stored in the `gradle.properties` along with the path to the key. Paths here need to be absolute, otherwise Gradle will look them up relative to the sub-projects (`android/`, `android/sample`, ...). + +Maven Central is managed by Sonatype. To sign up follow their [Getting Started](https://central.sonatype.org/pages/ossrh-guide.html) guide, which involves creating a JIRA account and opening an issue to apply for the `com.facebook` namespace. You will need to find an existing member of this namespace to vouch for you. While this is a bit of a task, it ensures that nobody from outside the organisation can publish under our name. -The `publish` (previously `uploadArchives`) gradle task uses the OSSRH Sonatype Nexus credentials to upload all Flipper Java artifacts. That -includes the core SDK as well as our plugins. The credentials are *not* your login to Nexus, but the user tokens -you can get from [your profile](https://oss.sonatype.org/#profile;User%20Token). +The `publish` (previously `uploadArchives`) gradle task uses the OSSRH Sonatype Nexus credentials to upload all Flipper Java artifacts. This includes the core SDK as well as our plugins. The credentials are *not* your login to Nexus, but the user tokens you can get from [your profile](https://oss.sonatype.org/#profile;User%20Token). -This is followed by the `closeAndReleaseRepository` gradle task, which is part of the -[`gradle-maven-publish-plugin`](https://github.com/vanniktech/gradle-maven-publish-plugin). It uses the credentials -to identify a "staging repository" and automatically close it. This staging repository is identified by the -[`SONATYPE_STAGING_PROFILE`](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/gradle.properties#L9) -property. Sonatype usually requires people to manually go to a web UI, verify that a given release is -complete and click some buttons. The plugin aims to do this for you. +This is followed by the `closeAndReleaseRepository` gradle task, which is part of the [gradle-maven-publish-plugin](https://github.com/vanniktech/gradle-maven-publish-plugin). It uses the credentials to identify a 'staging repository' and automatically close it. This staging repository is identified by the [SONATYPE_STAGING_PROFILE](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/gradle.properties#L9) property. Sonatype usually requires people to manually go to a web UI, verify that a given release is complete and click some buttons. The plugin aims to do this for you. #### Troubleshooting -There are a few parts which can go wrong here. +There are a few potential 'troubles': -- **Upload fails:** Maven Central is (at the time of writing) overloaded with projects migrating from - JCenter. The upload task attempts to retry but it can still time out. Manually re-running the job - through the GitHub UI should do the trick. -- **Closing fails:** Same as before, this can happen because of timeouts. -- **Retrying to close fails because of duplicate staging repositories:** This is particularly annoying - because you cannot fix this through automation. It happens when artifacts are uploaded multiple times - and now more than one staging repository exists. You must first *drop* (not close or release) the existing ones before - restarting the job. To do this, go to [Staging Repositories](https://oss.sonatype.org/#stagingRepositories), - select the open repositories and click "Drop". -- **NDK mismatch:** If Gradle complains about a missing NDK, this usually indicates that a dependency - has a hard requirement on a particular NDK. You can add it to the list in the `sdkmanager` command. -- **Artifacts not available:** Maven Central syncs with a delay of sometimes a few hours. You can - check directly on the [Maven2 main server](https://repo.maven.apache.org/maven2/com/facebook/flipper/) - if the artifacts with the new version number are uploaded. +* **Upload fails** - Maven Central is (at the time of writing) overloaded with projects migrating from JCenter. The upload task attempts to retry but it can still time out. Manually re-running the job through the GitHub UI should do the trick. +* **Closing fails** - as before, this can happen because of timeouts. +* **Retrying to close fails because of duplicate staging repositories** - particularly annoying because you can't fix this through automation. It happens when artifacts are uploaded multiple times and now more than one staging repository exists. You must first *drop* (not close or release) the existing ones before restarting the job. + + Take the following steps: + + * Go to [Staging Repositories](https://oss.sonatype.org/#stagingRepositories). + * Select the open repositories and click 'Drop'. + +* **NDK mismatch** - if Gradle complains about a missing NDK, this usually indicates that a dependency has a hard requirement on a particular NDK. You can add it to the list in the `sdkmanager` command. +* **Artifacts not available** - Maven Central syncs with a delay of sometimes a few hours. You can check directly on the [Maven2 main server](https://repo.maven.apache.org/maven2/com/facebook/flipper/) if the artifacts with the new version number are uploaded.