Summary: Follow-up to D38155922. This was done almost entirely by just following suggestions from the compiler. No behaviour changes, just fewer bit-copies in memory.
Reviewed By: lblasa
Differential Revision: D38199244
fbshipit-source-id: 777e9d833d25cb660307380ab88b12a1d4eedf09
Summary:
The limitation to two types is arbitrary and limiting. Instead, we want to be able to create as many cache artifacts as is sensible to improve the caching behaviour.
A lot of unnecessary cloning in here. I might optimise this a bit in the future but it's not really perf critical as this is highly I/O bound.
Reviewed By: lblasa
Differential Revision: D38155922
fbshipit-source-id: 78b86ebff54269c5049e59197f1c25fedfad0111
Summary: This is needed so we can roll out releases of Flipper Server for Linux.
Reviewed By: lawrencelomax
Differential Revision: D38155968
fbshipit-source-id: 2be032466cb8da867fa555fa545d1f236724461f
Summary: Upgrade to indicatif 0.16 and remove this as an exposed dep in the third-party repo.
Reviewed By: aigoncharov
Differential Revision: D36591356
fbshipit-source-id: e3e3d67f8522b94861b34422606b85afbebf669b
Summary: Take changes from D36140809 into account.
Reviewed By: aigoncharov
Differential Revision: D36206155
fbshipit-source-id: 794f8dd60e47f14acc4fb5fd9fd3d98fa5afdadf
Summary: Just this one for now as I can actually test that this works.
Reviewed By: lblasa
Differential Revision: D35902244
fbshipit-source-id: 9914c58968b4f9dfd25be7afaf6d89dffaad294a
Summary:
Just a bit of extra type safety. The previous diff caused a bit of a regression
by making things stringly-typed.
Reviewed By: lblasa
Differential Revision: D35902107
fbshipit-source-id: a599106f899ec3d205663b5791420aee29f3eeae
Summary: This allows us to extend this more easily to server builds, too.
Reviewed By: lblasa
Differential Revision: D35901750
fbshipit-source-id: 7030846ad485d709cf4c2e95f0cad287f98b051b
Summary:
This had collected a bit of dust and we will need to extend this a little for flipper-server.
- Update deps.
- Migrate away from deprecated clap2 model to derive (it's much shorter now and less imperative).
- ~~Some COW changes on how the progress bar crate handles strings.~~ (Reverted as this causes havvoc in `fbsource//third-party/rust`.)
- Upgraded to 2021 Edition.
Reviewed By: nikoant
Differential Revision: D35433571
fbshipit-source-id: ae0a91558610ae46069a5fc5162b524cde759454
Summary:
Best seen by looking at the packfile. This makes upgrades and changes
to the file format a lot more robust.
Reviewed By: priteshrnandgaonkar
Differential Revision: D27194152
fbshipit-source-id: a722adb3a59d73b7530287f7abd84ed865a35fa7
Summary:
Electron 12 broke packing again because one file was renamed. I'm now setting up a separate mode for using globs and ignores to create artifact bundles. This will work like a reverse gitignore file. However, to keep the logic simple, I'll keep the old mode for MacOS where the folder structure lends itself to comprehensive, exact lists.
**This doesn't actually change anything just yet apart from the "packfile" format. The next diff will add the new packing mode. Feedback is always welcome but there's no need for super close scrutiny.**
Reviewed By: mweststrate
Differential Revision: D27191506
fbshipit-source-id: 663cef8b93eef6c2dbb56ef66de51ea9551412dd
Summary: We're passing around PathBufs where Paths are enough and more efficient.
Reviewed By: fabiomassimo
Differential Revision: D27155300
fbshipit-source-id: cdf8d67e28a669130482d333c960d100bdaf0992
Summary:
Electron 12 renamed one .so and broke the build.
Time to implement T83339637.
Reviewed By: fabiomassimo
Differential Revision: D27155174
fbshipit-source-id: a034f79cacb5c27b62cc6b51c9899a3d3a227954
Summary:
Couldn't do this because we were stuck on an old version
of digest internally with breaking changes. Now there's a
`sha2-9` option to make it build.
Reviewed By: mweststrate
Differential Revision: D26404582
fbshipit-source-id: 9851de3439ea263eaddb558daaf4cf9e73b062cc
Summary: Exluded the license files because we don't need them.
Reviewed By: mweststrate
Differential Revision: D25995326
fbshipit-source-id: 8b42c69b75f33ca345d9e3b1310fef0fd928ea82
Summary:
This adds the definitions for how to split up Windows releases.
This only works with the previous change because otherwise
the Flipper.exe will change with every release build number.
Reviewed By: mweststrate
Differential Revision: D25977814
fbshipit-source-id: 4fb9d4e9562dedf43f9e9ec90a49801f60ed4550
Summary: Just going through all the lints that have popped up.
Reviewed By: jknoxville
Differential Revision: D24078571
fbshipit-source-id: 06504aafa969eea92ee934ac607d3daf51e47914
Summary: This now makes use of the `tarsum` mod to surface both intrinsic and extrinsic checksums as part of the manifest. I also threw in the file name and the bytes for good measure. This is output in Sandcastle logs and can help identify mismatches. The filename also means we no longer have to rely on piecing together paths on Sandcastle which is nice.
Reviewed By: nikoant
Differential Revision: D24046464
fbshipit-source-id: dc6dc50eb510a067674166b5fdb11af9cf6762b2
Summary:
# Problem
Oddly enough, since the addition of ReactiveCocoa to Electron on MacOS, we're seeing non-determinism when creating tar archives. P144049799 shows the difference between two release artifacts built on Sandcastle. Their contents are identical but they're packed differently.
# Approach
To counter this, I'm now calculating the checksum not based on the archive itself but on its *sorted* contents by first inserting all files into a `BTreeMap` with the file path as key and then lastly hashing all hashes, building some sort of bastardised Merkle tree.
# This diff
I'm only implementing the hashing here. The next diff will actually make use of this.
# Further steps
This requires a few more downstream changes which will require some finesse to roll out.
- The manifest will need to include both the intrinsic and extrinsic checksums as Launcher depends on the latter to verify the download integrity. We could also calculate the intrinsic checksum again, but that just adds more complexity which is hard to debug.
- Sandcastle will need to understand the new manifest format and we need to schedule the MSDK rollout accordingly. Not a huge problem as the call volume is low and manual.
- We need to modify the artifact and release ents to contain both checksums.
- The release endpoint needs to be modified to return the checksum the launcher cares about.
Reviewed By: nikoant
Differential Revision: D24024011
fbshipit-source-id: 55de748178c033c18a69c79c68f12e7c1aaf4deb
Summary:
Generated by formatting with rustfmt 2.0.0-rc.2 and then a second time with fbsource's current rustfmt (1.4.14).
This results in formatting for which rustfmt 1.4 is idempotent but is closer to the style of rustfmt 2.0, reducing the amount of code that will need to change atomically in that upgrade.
---
*Why now?* **:** The 1.x branch is no longer being developed and fixes like https://github.com/rust-lang/rustfmt/issues/4159 (which we need in fbcode) only land to the 2.0 branch.
---
Reviewed By: zertosh
Differential Revision: D23568784
fbshipit-source-id: 7401066bfc99390686eb3fd9dc6f9888ccef9487
Summary:
Compresses artifacts with LZMA by default. `--no-compression` can be used
to skip the quite time-intensive task to just quickly check if the archives
still add up to a full distribution.
Reviewed By: jknoxville
Differential Revision: D21456052
fbshipit-source-id: dfa4686f3fe7ce342b62dab4b5203bc0aad37d9a
Summary:
This does actually change from version to version as can be seen
when bundling both 0.40 and 0.41:
{F236391778}
Reviewed By: jknoxville
Differential Revision: D21426528
fbshipit-source-id: 53c0565f7693010a2096088af1829eab210a75cc
Summary: Add progress bars. The previous change broke the display as we'd write to the console in parallel, too.
Reviewed By: jknoxville
Differential Revision: D21405230
fbshipit-source-id: 3a60690c216fd2f44816ef1b01917f66ebe804df
Summary: This is to make parallelising this in the next step easier.
Reviewed By: jknoxville
Differential Revision: D21405192
fbshipit-source-id: b1f2841526a65fcbf545feb69f258874c2134603
Summary: When file writing fails, attach some context in the error message.
Reviewed By: jknoxville
Differential Revision: D21400273
fbshipit-source-id: f7cce5446ec27d5eb38f1902f24c61169dd644ac
Summary: Hashes the artifacts and adds the information to a manifest. This will then be used to upload to a blob store, and save the information so it can be retrieved again later. Not sure if I want to stick with SHA256 or if there's something more efficient like Cityhash, given that we don't really care about cryptographic properties here.
Reviewed By: jknoxville
Differential Revision: D21380369
fbshipit-source-id: e9c5cd56d94f3083ae5ed6396673d00cbf98ce39
Summary:
Instead of hardcoding the targets, it will now build all specified
types in the "packlist".
Reviewed By: jknoxville
Differential Revision: D21349762
fbshipit-source-id: 58f4a3bbf0b6ff4dd87eb44bbd7b200127da8017
Summary:
This is a nice solution if you don't want to spend
too much time thinking about error handling.
You have one common return type and can basically use `?`
everywhere while still maintaining the flexibility to
create custom error types where needed.
Reviewed By: jknoxville
Differential Revision: D21349046
fbshipit-source-id: 073539ce8422cdb3e0141886e95321052bc0c7a3
Summary:
This is a tool (right now more a script, really) for splitting a Flipper distribution into smaller chunks that can be individually cached, hashed and downloaded. It's incomplete but can already bundle up Mac apps into smaller chunks that can be merged back again into one app.
It takes a config file that specifies how to slice the inputs up and comes with a default copy bundled in. That's in line with the general "design" where it tries to make the right assumptions, for instance defaulting to the normal dist output directory.
Reviewed By: jknoxville
Differential Revision: D21302551
fbshipit-source-id: b22670ff4825ce526dd0f20320f87080c058cd93
