public-releases.mdx (Under the Hood - Public Flipper Releases)
Summary: Restyle of page, including changes to spelling, grammar, links, and structure (where relevant). Reviewed By: passy Differential Revision: D36666339 fbshipit-source-id: 4c57746bd62fd52fa8dff4fda3098c443941ae30
This commit is contained in:
Kevin Strider
committed by
Facebook GitHub Bot
Facebook GitHub Bot
parent
23f73f0b1e
commit
50919e26f5
@@ -4,161 +4,127 @@ title: Public Flipper Releases
|
||||
---
|
||||
import useBaseUrl from '@docusaurus/useBaseUrl';
|
||||
|
||||
This document outlines how Flipper releases work on GitHub.
|
||||
This page outlines how Flipper releases work on GitHub.
|
||||
|
||||
## Kick-Off
|
||||
|
||||
A release is kicked off by a special commit that has a subject with the format
|
||||
`Flipper Release vX.Y.Z`, e.g. [4fa2c9761](https://github.com/facebook/flipper/commit/4fa2c9761a8359c65ccc62fee76490572616f0c1).
|
||||
`Flipper Release vX.Y.Z` (see the GitHub [example](https://github.com/facebook/flipper/commit/4fa2c9761a8359c65ccc62fee76490572616f0c1)).
|
||||
|
||||
This is triggered from a bot within Facebook that runs [prepare-release.sh](https://github.com/facebook/flipper/blob/main/scripts/prepare-release.sh)
|
||||
but the only thing special about the commit is its subject line. Anyone could run the
|
||||
script and would kick off the remaining jobs once the commit lands in main.
|
||||
This is triggered from a bot within Facebook that runs [prepare-release.sh](https://github.com/facebook/flipper/blob/main/scripts/prepare-release.sh). The only thing special about the commit is its subject line. Anyone could run the script and kick off the remaining jobs once the commit lands in main.
|
||||
|
||||
The commit bumps the version of Flipper Desktop as well as various SDK components
|
||||
and libraries that are to be published to npm and other package repositories.
|
||||
The commit bumps the version of Flipper Desktop as well as various SDK components and libraries that are to be published to npm and other package repositories.
|
||||
|
||||
Importantly, it is immediately followed by a "SNAPSHOT" commit (e.g. [02a56da3f](https://github.com/facebook/flipper/commit/02a56da3f5440b35616604f73167307319cca48f)) which sets
|
||||
the version of our Java dependencies to `X.Y.(Z+1)-SNAPSHOT` (that's the
|
||||
patch version incremented by one and a `-SNAPSHOT` suffix appended).
|
||||
That's a weird Maven-ism which allows us to continuously publish snapshot
|
||||
releases from the main branch.
|
||||
Importantly, it is immediately followed by a 'SNAPSHOT' commit (see the GitHub [example](https://github.com/facebook/flipper/commit/02a56da3f5440b35616604f73167307319cca48f)), which sets the version of our Java dependencies to `X.Y.(Z+1)-SNAPSHOT` (that's the patch version incremented by one and a `-SNAPSHOT` suffix appended). It's a weird Maven-ism that enables us to continuously publish snapshot releases from the main branch.
|
||||
|
||||
## Desktop Release
|
||||
|
||||
The release process for the desktop app is entirely driven by [GitHub Actions](https://github.com/features/actions).
|
||||
The entry point is the [release.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/release.yml) workflow
|
||||
which is triggered by changes to the `desktop/package.json` file on the main branch.
|
||||
The entry point is the [release.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/release.yml) workflow, which is triggered by changes to the `desktop/package.json` file on the main branch.
|
||||
|
||||
GitHub Actions has a limitation we need to work around here: It allows push events to as triggers for
|
||||
a workflow, but not individual commits. This requires us to first scan through all potential commits
|
||||
of a push to identify commits that match the aforementioned versioning schema.
|
||||
This is done through a [custom action](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.github/workflows/release.yml#L15-L18).
|
||||
GitHub Actions has a limitation that we need to work around: it enables push events to act as triggers for a workflow, but not individual commits. This requires us to first scan through all potential commits of a push to identify commits that match the aforementioned versioning schema. This is done through a [custom action](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.github/workflows/release.yml#L15-L18).
|
||||
|
||||
Only if this action is successful, the remaining steps are taken which
|
||||
If this action is successful, the remaining steps are taken:
|
||||
|
||||
- Check out the versioning commit.
|
||||
- Create a git tag for it.
|
||||
- Create a GitHub release for the tag.
|
||||
- Build releases for Linux, Mac and Windows.
|
||||
- Upload all these releases to temporary storage.
|
||||
- Download the release artifacts and attach them to the GitHub release.
|
||||
- Dispatch to separate workflows for publishing Cocoapods and npm packages (see below).
|
||||
* Check out the versioning commit.
|
||||
* Create a git tag for it.
|
||||
* Create a GitHub release for the tag.
|
||||
* Build releases for Linux, Mac and Windows.
|
||||
* Upload all these releases to temporary storage.
|
||||
* Download the release artifacts and attach them to the GitHub release.
|
||||
* Dispatch to separate workflows for publishing Cocoapods and npm packages (see below).
|
||||
|
||||
## iOS Release
|
||||
|
||||
iOS releases are run in GitHub Actions but exist as a separate workflow. They can be triggered in three ways:
|
||||
iOS releases are run in [GitHub Actions](https://github.com/features/actions) but exist as a separate workflow. They can be triggered in three ways:
|
||||
|
||||
1. When a tag is pushed.
|
||||
2. By manually triggering the workflow (see below).
|
||||
3. Through a `dispatch_workflow` event which is issued as a last step of the desktop release process.
|
||||
2. By manually triggering the workflow (see the following screenshot).
|
||||
3. Through a `dispatch_workflow` event that is issued as a last step of the desktop release process.
|
||||
|
||||
<img alt="Manually triggering a workflow" src={useBaseUrl("img/trigger-publish-pod-workflow.png")} />
|
||||
|
||||
[The workflow](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-pods.yml) follows the default Cocoapods update procedure, bumps and published both the Flipper and FlipperKit pod
|
||||
and finally creates a [pull request](https://github.com/facebook/flipper/pull/1754) containing the updated references. This PR must be manually merged.
|
||||
The [workflow](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-pods.yml) follows the default Cocoapods update procedure, bumps and publishes both the Flipper and FlipperKit pod, then creates a [Pull Request](https://github.com/facebook/flipper/pull/1754) (PR) containing the updated references. This PR must be manually merged.
|
||||
|
||||
:::note
|
||||
**Authentication** is managed through the secret environment variable `COCOAPODS_TRUNK_TOKEN`.
|
||||
:::
|
||||
|
||||
## npm releases
|
||||
|
||||
The JavaScript libraries published as part of a Flipper release closely follow the iOS release procedure.
|
||||
Same as before, it is a workflow that is triggered by one of the three events, which should in all but
|
||||
exceptional circumstances be a dispatch event. The workflow is defined in [publish-npm.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-npm.yml).
|
||||
As with the [iOS release](#ios-release), it's a workflow that is triggered by one of the three events, which should, in all but exceptional circumstances, be a dispatch event.
|
||||
The workflow is defined in [publish-npm.yml](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-npm.yml).
|
||||
|
||||
From there, we use a [script](https://github.com/facebook/flipper/blob/main/desktop/scripts/bump-versions.ts) to
|
||||
bump the versions of our Yarn workspaces, and publish all public packages (`flipper`, `flipper-babel-transformer`, ...)
|
||||
and our React Native bindings.
|
||||
From there, we use a [script](https://github.com/facebook/flipper/blob/main/desktop/scripts/bump-versions.ts) to bump the versions of our Yarn workspaces, and publish all public packages (`flipper`, `flipper-babel-transformer`, ...) and our React Native bindings.
|
||||
|
||||
:::note
|
||||
The **authentication** to npm is managed by a secret environment variable called `FLIPPER_NPM_TOKEN`.
|
||||
:::
|
||||
|
||||
## Android Release
|
||||
|
||||
Android has three types of jobs currently running. The `snapshot` job is an
|
||||
outlier in that it still runs on CircleCI. This gives us some additional
|
||||
capacity as these jobs can take quite a while and the occasional failure
|
||||
due to timeouts or network errors isn't a dealbreaker.
|
||||
Android has three types of jobs currently running:
|
||||
|
||||
The three jobs are:
|
||||
1. [snapshot](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) - runs on every commit on the main branch and publishes 'SNAPSHOT' releases to Maven Central. It runs on CircleCI.
|
||||
2. [publish-android](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-android.yml) - usually triggered by a `dispatch_workflow` event. It uploads our Java artifacts to Maven Central and attaches the Android sample app to the release page on GitHub. It runs on GitHub Actions.
|
||||
3. [android-sample](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) - runs on every push and open pull request. It builds the sample and tutorial apps and uploads the sample APK as artifact for easy debugging and testing.
|
||||
|
||||
1. The [`snapshot`](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) job runs on every commit on the main branch and publishes "SNAPSHOT" releases to Maven Central. It runs on CircleCI.
|
||||
2. The [`publish-android`](https://github.com/facebook/flipper/blob/main/.github/workflows/publish-android.yml) job is usually triggered by a `dispatch_workflow` event. It uploads our Java artifacts to Maven Central and attaches the Android sample app to the release page on GitHub. It runs on GitHub Actions.
|
||||
3. The [`android-sample`](https://github.com/facebook/flipper/blob/main/.github/workflows/android-sample.yml) job runs on every push and open pull request. It builds the sample and tutorial apps and uploads the sample APK as artifact for easy debugging and testing.
|
||||
The `snapshot` job is an outlier in that it still runs on CircleCI. This gives us some additional capacity as these jobs can take quite a while and the occasional failure due to timeouts or network errors isn't a dealbreaker.
|
||||
|
||||
### CircleCI Configuration
|
||||
|
||||
The Android snapshot build is run on
|
||||
[CircleCI](https://app.circleci.com/pipelines/github/facebook/flipper?branch=main)
|
||||
and configured in
|
||||
[`.circleci/config.yml`](https://github.com/facebook/flipper/blob/main/.circleci/config.yml)
|
||||
The Android snapshot build is run on [CircleCI](https://app.circleci.com/pipelines/github/facebook/flipper?branch=main) and configured in [.circleci/config.yml](https://github.com/facebook/flipper/blob/main/.circleci/config.yml).
|
||||
|
||||
There are two potential points for breakage:
|
||||
|
||||
1. The [base image](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.circleci/config.yml#L5) used in the build instructions refers to a specific SDK version and requires occasional updating.
|
||||
2. The [platform installation](https://github.com/facebook/flipper/blob/b5e613141e98528f45d3d2864e08278b1c7d4973/.circleci/config.yml#L23) through the `sdkmanager` tool of the Android SDK may require additional SDKs or NDKs to be installed if they're not part of the base image.
|
||||
1. [Base image](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/.circleci/config.yml#L5) - used in the build instructions refers to a specific SDK version and requires occasional updating.
|
||||
2. [Platform installation](https://github.com/facebook/flipper/blob/b5e613141e98528f45d3d2864e08278b1c7d4973/.circleci/config.yml#L23) - runs through the `sdkmanager` tool of the Android SDK. It may require additional SDKs or NDKs to be installed if they're not part of the base image.
|
||||
|
||||
One non-obvious aspect is that of **authentication** for uploads. The repository contains a symmetrically encrypted
|
||||
copy of our credentials to Sonatype (for Maven Central). The [snapshot release script](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/scripts/publish-android-snapshot.sh)
|
||||
decodes the file on the fly by using a secret Circle CI exposes through an environment variable.
|
||||
One non-obvious aspect is that of **authentication** for uploads. The repository contains a symmetrically encrypted copy of our credentials to Sonatype (for Maven Central). The [snapshot release script](https://github.com/facebook/flipper/blob/1cad57d75077bdb3594e33e845ab6d226c5d2c86/scripts/publish-android-snapshot.sh) decodes the file on the fly by using a secret Circle CI exposes through an environment variable.
|
||||
|
||||
### GitHub Action Workflow
|
||||
|
||||
As with the iOS release before, the workflow for Android releases is triggered by three types of events:
|
||||
As with the [iOS release](#ios-release) (see above), the workflow for Android releases is triggered by three types of events:
|
||||
|
||||
1. When a tag is pushed.
|
||||
2. By manually triggering the workflow.
|
||||
3. Through a `dispatch_workflow` event which is issued as a last step of the desktop release process.
|
||||
3. Through a `dispatch_workflow` event, which is issued as a last step of the desktop release process.
|
||||
|
||||
In normal circumstances, the third event will kick off an Android release build.
|
||||
The workflow is defined in [`publish-android.yml`](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/.github/workflows/publish-android.yml).
|
||||
In normal circumstances, the third event will kick off an Android release build. The workflow is defined in [publish-android.yml](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/.github/workflows/publish-android.yml) in GitHub.
|
||||
|
||||
We first install two NDK versions that are required by our dependencies. To publish release artifacts
|
||||
(i.e. non-`SNAPSHOT` artifacts), Maven Central requires them to be signed with a GnuPG key. The
|
||||
only requirement about the key itself is, that it needs to be exported to a Keyserver. Ours
|
||||
is published [to the Ubuntu Keyserver](https://keyserver.ubuntu.com/pks/lookup?search=Flipper+Bot+%28I+sign+Flipper+releases%29+%3Crealpassy%40fb.com%3E&fingerprint=on&op=index).
|
||||
We first install two NDK versions that are required by our dependencies. To publish release artifacts (non-`SNAPSHOT` artifacts), that Maven Central, requires them to be signed with a GnuPG key. The
|
||||
only requirement about the key is that it needs to be exported to a Keyserver. Ours is published to the [Ubuntu Keyserver](https://keyserver.ubuntu.com/pks/lookup?search=Flipper+Bot+%28I+sign+Flipper+releases%29+%3Crealpassy%40fb.com%3E&fingerprint=on&op=index).
|
||||
|
||||
To publish your own key, run
|
||||
To publish your own key, run the following:
|
||||
|
||||
```bash
|
||||
gpg --send-keys --keyserver keyserver.ubuntu.com <KEY_ID>
|
||||
```
|
||||
|
||||
For the initial setup, the secret keyring was exported as `gpg2 --export-secret-keys <secret_key_id> | base64` and stored
|
||||
as secret on GitHub with the name `GPG_KEY_CONTENTS`. As part of the workflow, it is written to disk after reversing the base64 encoding.
|
||||
The key id and key password are subsequently stored in the `gradle.properties` along with the path to the key. Paths here need to be
|
||||
absolute, otherwise Gradle will look them up relative to the sub-projects (`android/`, `android/sample`, ...).
|
||||
For the initial setup, the secret keyring was exported as `gpg2 --export-secret-keys <secret_key_id> | base64` and stored as a secret on GitHub with the name `GPG_KEY_CONTENTS`.
|
||||
|
||||
Maven Central is managed by Sonatype. To sign up follow [their guide](https://central.sonatype.org/pages/ossrh-guide.html) which
|
||||
involves creating a JIRA account and opening an issue to apply for the `com.facebook` namespace. You will need
|
||||
to find an existing member of this namespace to vouch for you. While this is a lot, it ensures that nobody
|
||||
As part of the workflow, it is written to disk after reversing the base64 encoding. The key id and key password are subsequently stored in the `gradle.properties` along with the path to the key. Paths here need to be absolute, otherwise Gradle will look them up relative to the sub-projects (`android/`, `android/sample`, ...).
|
||||
|
||||
Maven Central is managed by Sonatype. To sign up follow their [Getting Started](https://central.sonatype.org/pages/ossrh-guide.html) guide, which involves creating a JIRA account and opening an issue to apply for the `com.facebook` namespace. You will need to find an existing member of this namespace to vouch for you. While this is a bit of a task, it ensures that nobody
|
||||
from outside the organisation can publish under our name.
|
||||
|
||||
The `publish` (previously `uploadArchives`) gradle task uses the OSSRH Sonatype Nexus credentials to upload all Flipper Java artifacts. That
|
||||
includes the core SDK as well as our plugins. The credentials are *not* your login to Nexus, but the user tokens
|
||||
you can get from [your profile](https://oss.sonatype.org/#profile;User%20Token).
|
||||
The `publish` (previously `uploadArchives`) gradle task uses the OSSRH Sonatype Nexus credentials to upload all Flipper Java artifacts. This includes the core SDK as well as our plugins. The credentials are *not* your login to Nexus, but the user tokens you can get from [your profile](https://oss.sonatype.org/#profile;User%20Token).
|
||||
|
||||
This is followed by the `closeAndReleaseRepository` gradle task, which is part of the
|
||||
[`gradle-maven-publish-plugin`](https://github.com/vanniktech/gradle-maven-publish-plugin). It uses the credentials
|
||||
to identify a "staging repository" and automatically close it. This staging repository is identified by the
|
||||
[`SONATYPE_STAGING_PROFILE`](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/gradle.properties#L9)
|
||||
property. Sonatype usually requires people to manually go to a web UI, verify that a given release is
|
||||
complete and click some buttons. The plugin aims to do this for you.
|
||||
This is followed by the `closeAndReleaseRepository` gradle task, which is part of the [gradle-maven-publish-plugin](https://github.com/vanniktech/gradle-maven-publish-plugin). It uses the credentials to identify a 'staging repository' and automatically close it. This staging repository is identified by the [SONATYPE_STAGING_PROFILE](https://github.com/facebook/flipper/blob/8d5f136a349e77ec9ccebd303054e0e142cbab30/gradle.properties#L9) property. Sonatype usually requires people to manually go to a web UI, verify that a given release is complete and click some buttons. The plugin aims to do this for you.
|
||||
|
||||
#### Troubleshooting
|
||||
|
||||
There are a few parts which can go wrong here.
|
||||
There are a few potential 'troubles':
|
||||
|
||||
- **Upload fails:** Maven Central is (at the time of writing) overloaded with projects migrating from
|
||||
JCenter. The upload task attempts to retry but it can still time out. Manually re-running the job
|
||||
through the GitHub UI should do the trick.
|
||||
- **Closing fails:** Same as before, this can happen because of timeouts.
|
||||
- **Retrying to close fails because of duplicate staging repositories:** This is particularly annoying
|
||||
because you cannot fix this through automation. It happens when artifacts are uploaded multiple times
|
||||
and now more than one staging repository exists. You must first *drop* (not close or release) the existing ones before
|
||||
restarting the job. To do this, go to [Staging Repositories](https://oss.sonatype.org/#stagingRepositories),
|
||||
select the open repositories and click "Drop".
|
||||
- **NDK mismatch:** If Gradle complains about a missing NDK, this usually indicates that a dependency
|
||||
has a hard requirement on a particular NDK. You can add it to the list in the `sdkmanager` command.
|
||||
- **Artifacts not available:** Maven Central syncs with a delay of sometimes a few hours. You can
|
||||
check directly on the [Maven2 main server](https://repo.maven.apache.org/maven2/com/facebook/flipper/)
|
||||
if the artifacts with the new version number are uploaded.
|
||||
* **Upload fails** - Maven Central is (at the time of writing) overloaded with projects migrating from JCenter. The upload task attempts to retry but it can still time out. Manually re-running the job through the GitHub UI should do the trick.
|
||||
* **Closing fails** - as before, this can happen because of timeouts.
|
||||
* **Retrying to close fails because of duplicate staging repositories** - particularly annoying because you can't fix this through automation. It happens when artifacts are uploaded multiple times and now more than one staging repository exists. You must first *drop* (not close or release) the existing ones before restarting the job.
|
||||
|
||||
Take the following steps:
|
||||
|
||||
* Go to [Staging Repositories](https://oss.sonatype.org/#stagingRepositories).
|
||||
* Select the open repositories and click 'Drop'.
|
||||
|
||||
* **NDK mismatch** - if Gradle complains about a missing NDK, this usually indicates that a dependency has a hard requirement on a particular NDK. You can add it to the list in the `sdkmanager` command.
|
||||
* **Artifacts not available** - Maven Central syncs with a delay of sometimes a few hours. You can check directly on the [Maven2 main server](https://repo.maven.apache.org/maven2/com/facebook/flipper/) if the artifacts with the new version number are uploaded.
|
||||
|
||||
Reference in New Issue
Block a user