Summary: Make sure flipper-server is bundled in such a way that it is self-contained NPX-able. Also added some checks to make sure that dev dependencies don't accidentallly end up in in Flipper buidls
Reviewed By: nikoant
Differential Revision: D33190254
fbshipit-source-id: 443162e537d8ca9f956acac2d7bd52cbf0c92172
Summary:
This diff makes flipper-server stand-alone servable as a single package.
It basically works as follows:
- (default) plugins are build as usually into static folder
- static folder is copied into flipper-server/static (as far as relevant)
- `flipper-server/src/index.tsx` is bundled into `flipper-server/dist/index.js`
- `flipper-ui-browser/src/index.tsx` is bundled into `flipper-server/static/bundle.js`
If flipper-server is started without the `--bundler` config, the `bundle.js` will be served statically, rather than starting up the Metro bundler with all the typical transforms (in a distributed version of the package those all wouldn't resolve)
Things to be done in next diffs:
* make sure plugins actually load in the non bundled setup
* make sure flipper-server gets published properly
* make sure build is created as part of CI
At the end of this stack, running `npx flipper-server` on any machine should basically be a viable approach to get flipper up and running locally :)
Reviewed By: nikoant
Differential Revision: D33171107
fbshipit-source-id: 4af8ac2699467a0b55283fe084640482700744c2
Summary:
Pull Request resolved: https://github.com/facebook/flipper/pull/3188
Fixes failing typescript issue (probably case sensitivity thing, on windows it was complaining about `Metro` not being resolvable while we use `metro` everywhere). Put a `ts-ignore` on it, since the metro typings we use are empty anyway
Disabled a test that failed consistently on GH CI. Maybe timing or Node version issue?
Reviewed By: fabiomassimo
Differential Revision: D33191515
fbshipit-source-id: 18a143024824eeeafaffe4941df474591fb7b70a
Summary: This diff is some boyscouting on cleaning up our globals, and stop using them inconsistently icmw global or window
Reviewed By: aigoncharov
Differential Revision: D33171108
fbshipit-source-id: 400893e5f31523631a7ab6fda428524c751901f1
Summary:
This diff adds `types` fields on the compiler config for every project. This way we can make sure that for example node types and packages are not available in flipper-ui-core. Without an explicit types field, all types would be shared between all packages, and implicitly included into the compilation of everything. For the same reason `types/index.d.ts` has been removed, we want to be intentional on which types are being used in which package.
This diff does most of the work, the next diff will fine tune the globals, and do some further cleanup.
As an alternative solution I first tried a `nohoist: **/node_modules/types/**` and make sure every package list explicitly the types used in package json, which works but is much more error prone, as for example two different react types versions in two packages will cause the most unreadable compiler error due to the types not being shared and not literally the same.
Reviewed By: lawrencelomax
Differential Revision: D33124441
fbshipit-source-id: c2b9d768f845ac28005d8331ef5fa1066c7e4cd7
Summary:
This diff removes most deps from the root package.json, which now only contains electron and shared build / test infra structure: lint, prettier, jest, typescript.
This makes it possible to control much better which packages are used where, as all sub packages now have their deps explicitly in their package.json instead of incidentally shared. This allows for example to disable DOM types for all packages by default (flipper-plugin, ui(-core) and app still request it), and in the next diff I hope to add to this that nodeJS types are no longer shared either, so that UI oriented packages will generate compile errors when using Node built-ins
This diff removes most deps that were currently unused, and dedupes a bunch of other ones, so the build should probably be a bit smaller now as well:
{F686704253}
{F686704295}
Reviewed By: antonk52
Differential Revision: D33062859
fbshipit-source-id: 5afaa4f2103d055188382a3370c1fffa295a298a
Summary:
Bundled plugins so far didn't load because the defaultPlugins was not generated yet. Fixed follow up errors that resulted from node modules not being available in the browser, and made the process more robust so that one plugin that fails to initialise doesn't kill all bundled plugins from being loaded.
Cleaned up the server scripts, and reused the BUILTINS list that is already maintained in the babel transformer.
Report errors during the build process if modules are referred that are stubbed out (later on we could maybe error on that)
Reviewed By: aigoncharov
Differential Revision: D33020243
fbshipit-source-id: 3ce13b0049664b5fb19c1f45f0b33c1d7fdbea4c
Summary:
The build process for the server was a simple ts-node that compiled all deps. However, that didn't do any source transformations we need, like replacing `fb-stubs` with `fb` in facebook builds.
This diff works out the dev build process to align more with how other parts of the code base is build, by starting metro to build and bundle the server (only the sources of flipper-server, flipper-server-core and other flipper packages are bundled, node-deps are left as is).
To achieve this, since metro doesn't have support for 'external' packages like any arbitrarily other bundler, we recycle the electronRequire work around that is used in the desktop app as well
Reviewed By: aigoncharov
Differential Revision: D32949677
fbshipit-source-id: 00d326bb17b68aece6fb43af98d0def13b335e74
Summary: Moved all types related to plugin descriptions from plugin-lib (which handles downloads and such) to flipper-common. The goal of that is to remove all plugin-lib usage from ui-core to server-core, so that the UI itself doesn't do any file operations anymore related to plugins. That will be done in next diffs, this just moves types but no code.
Reviewed By: nikoant, aigoncharov
Differential Revision: D32665064
fbshipit-source-id: 86d908e7264569b0229b09290a891171876c8e00
Summary: Remove the duplicate function and centralise it as nikoant suggested.
Reviewed By: fabiomassimo
Differential Revision: D29548480
fbshipit-source-id: 3e931cc88198415017c557c6b7c81cb35c3f22c9
Summary: This makes it possible to save source maps to a separate folder so we can upload them from CI.
Reviewed By: nikoant
Differential Revision: D29521599
fbshipit-source-id: a137659092b7648858b97ecf5b5c60c88889517a
Summary: This change allows passing a directory containing plugin packages to include them into Flipper distributive. This is used by release command on Sandcastle (see D28626715).
Reviewed By: passy
Differential Revision: D28628425
fbshipit-source-id: 9c5d7527721f99b43991bace0b5e2f3a4ede0d13
Summary:
Sorry for long diff! I can try to split it if necessary, but many changes here are 1-1 replacements / renames.
**Preambule**
Currently we bundle default plugins into the Flipper main bundle. This helps us to reduce bundle size, because of plugin dependencies re-use. E.g. if multiple plugins use "lodash" when they are bundled together, only one copy of "lodash" added. When they are bundled separately, the same dependency might be added to each of them. However as we're not going to include most of plugins into Flipper distributive anymore and going to rely on Marketplace instead, this bundling doesn't provide significant size benefits anymore. In addition to that, bundling makes it impossible to differentiate whether thrown errors are originated from Flipper core or one of its plugins.
Why don't we remove plugin bundling at all? Because for "dev mode" it actually quite useful. It makes dev build start much faster and also enables using of Fast Refresh for plugin development (fast refresh won't work for plugins loaded from disk).
**Changes**
This diff introduces new option "no-bundled-plugins" for "yarn start" and "yarn build" commands. For now, by default, we will continue bundling default plugins into the Flipper main bundle, but if this option provided then we will build each default plugin separately and include their packages into the Flipper distributive as "pre-installed" to be able to load them from disk even without access to Marketplace.
For "yarn start", we're adding symlinks to plugin folders in "static/defaultPlugins" and then they are loaded by Flipper. For "yarn build" we are dereferencing these symlinks to include physical files of plugins into folder "defaultPlugins" of the produced distributive. Folder "defaultPlugins" is excluded from asar, because loading of plugins from asar archive might introduce some unexpected issues depending on their implementation.
Reviewed By: mweststrate
Differential Revision: D28431838
fbshipit-source-id: f7757e9f5ba9183ed918d70252de3ce0e823177d
Summary:
This is not doing much in terms of size. We're talking 780 bytes after
PACK compression but we're losing a lot of information in our errors
(before have symbolication).
We may reenable this at some point but I think it might be safe to leave it to this.
Reviewed By: nikoant
Differential Revision: D27887636
fbshipit-source-id: 15affeac588db667914320cbd098f7041b8b9c09
Summary: Currently when babel transformation is failed, only stack is shown. It's better to log the whole error object because it also contains path and line number of the code failing to transpile.
Reviewed By: passy
Differential Revision: D27034942
fbshipit-source-id: 76e03871c924489e2698a26622e6bc6fdd305560
Summary: Patch was always set to "0" for open source releases even if it is set to "1" in package.json
Reviewed By: passy
Differential Revision: D26975947
fbshipit-source-id: b3f127c9746797fbb5a79dfcd9bd52210c3a7df8
Summary:
Renamed actions "star" and "unstar" everywhere to "enable", "disable" and "switch". The logic behind original "star" action changed significantly, so this rename just makes everything much clearer.
Please note that as a part of rename persisted state fields "userStarredPlugins" and "userStarredDevicePlugins" were renamed. I've added a "redux-persist" migration for seamless transition.
Reviewed By: passy
Differential Revision: D26606459
fbshipit-source-id: 83ad475f9b0231194701c40a2cdbda36f02c3d10
Summary: Now as we can install device plugins from Marketplace, I was able to unbundle device plugins which were used quite rarely, but still added ~3 MB to the final Flipper bundle.
Reviewed By: mweststrate
Differential Revision: D26337297
fbshipit-source-id: 23a36cc15ca976a1215440192718d985e1f322fd
Summary: Unbundle most of plugins from Insiders version of Flipper. Users will need to install them from Marketplace.
Reviewed By: passy
Differential Revision: D25558043
fbshipit-source-id: 648b0d4c83d9096972b5463cdcaa3de23426bdda
Summary:
I've re-designed interfaces describing plugins as I found that mental overhead working with them became too expensive because of slightly flawed design. However this cascaded changes in many files so you can see how extensively these interfaces used in our codebase.
Before this change we had one interface PluginDetails which described three different entities: 1) plugins installed on the disk 2) plugins bundled into Flipper 3) plugins available on Marketplace. It's hard to use this "general" PluginDetails interface because of this as you always need to think about all three use cases everywhere.
After this change we have 3 separate interfaces: InstalledPluginDetails, BundledPluginDetails and DownloadablePluginDetails and things became much type-safer now.
Reviewed By: mweststrate
Differential Revision: D25530383
fbshipit-source-id: b93593916a980c04e36dc6ffa168797645a0ff9c
Summary: We have some custom parsing here and there and I'm going to add some more args, so decided it's better to cleanup this a bit.
Reviewed By: jknoxville
Differential Revision: D25398421
fbshipit-source-id: 9d43029eef07648d0b01590e9cf7e7fe400b31d0
Summary:
This diff sets all package version to "0.0.0" except of the root package and changes the bump script to only bump version in the root package. This should reduce possibility of conflicts on release diffs. Anyway we always use the same version for all of our packages, so we can only set it to the root.
Before npm publishing we will set all package versions to the same number as in the root package (we actually already do that) so there will be no differences except we won't need to bump version in more than 100 packages each release.
Reviewed By: mweststrate
Differential Revision: D25162373
fbshipit-source-id: 02fe401bee72845339c67925c130027bdaee559d
Summary: Just rolling back a part of D24782300 (dce7422ba6) where I've accidently removed using of unique number because did not understood how it is passed and wrongly decided it is not needed.
Reviewed By: passy
Differential Revision: D25027458
fbshipit-source-id: f623732c161bca01859f91444c2a0ed211a5fad6
Summary: Scripts now accept full version instead of only patch part of it.
Reviewed By: jknoxville
Differential Revision: D24782300
fbshipit-source-id: 1c454beb9640a5b38669fe39eab642245b0f0f91
Summary:
- Removed compilation on startup which is not required anymore after switching to plugin spec v2.
- Removed from Node process ("static" package) all the dependencies which are not required anymore: e.g. metro, babel etc.
- Plugin loading code from node process moved to browser process and made asyncronous.
Some expected benefits after these changes:
1) Reduced size of Flipper bundle (~4.5MB reduction for lzma package in my tests) as well as startup time. It's hard to say the exact startup time difference as it is very machine-dependent, and on my machine it was already fast ~1500ms (vs 5500ms for p95) and decreased by just 100ms. But I think we should definitely see some improvements on "launch time" analytics graph for p95/p99.
2) Plugin loading is async now and happens when UI is already shown, so perceptive startup time should be also better now.
3) All plugin loading code is now consolidated in "app/dispatcher/plugins.tsx" instead of being splitted between Node and Browser processes as before. So it will be easier to debug plugin loading.
4) Now it is possible to apply updates of plugins by simple refresh of browser window instead of full Electron process restart as before.
5) 60% less code in Node process. This is good because it is harder to debug changes in Node process than in Browser process, especially taking in account differences between dev/release builds. Because of this Node process often ended up broken after changes. Hopefully now it will be more stable.
Changelog: changed the way of plugin loading, and removed obsolete dependencies, which should reduce bundle size and startup time.
Reviewed By: passy
Differential Revision: D23682756
fbshipit-source-id: 8445c877234b41c73853cebe585e2fdb1638b2c9
Summary: Disabled producing inline source maps for debug builds. Source maps are already writte to "map" files, so this does not make sense to also inline them to the js bundle. Passing "sourceMapUrl" parameter to Metro fixed this.
Reviewed By: mweststrate
Differential Revision: D22159501
fbshipit-source-id: f06225cd43f44a7f93afcc32bb0501323b35d0c3
Summary:
Disabled source maps for Flipper release builds.
1) I found that source maps was not only written to the "bundle.map" file, but also inlined into "bundle.js" in base64 format which hugely affected the total bundle size.
2) In addition to the size, these inlined source maps were also one of 2 causes that Flipper package size was not deterministic and changed so significantly in every diff even when Flipper was not touched at all.
Changelog: Disabled source maps in Flipper release builds thus reducing size by ~20%.
Reviewed By: mweststrate
Differential Revision: D22158898
fbshipit-source-id: b35cfba80553d01aec256983c3ba772ed0534d8f
Summary: Use interface PluginDetails everywhere where plugins are handled and removed PluginDefinition type which was effectively a subset of PluginDetails
Reviewed By: mweststrate
Differential Revision: D21927456
fbshipit-source-id: 434ebeef955b922cc11757e78fbba8dec05f1060
Summary: Load either installed or bundled version of plugin depending on which is newer.
Reviewed By: mweststrate
Differential Revision: D21858965
fbshipit-source-id: aa46eafe0b5137134fadad827749672441f2c9e5
Summary: This is required for the new release of promisify-child-process.
Reviewed By: mweststrate
Differential Revision: D21543812
fbshipit-source-id: dc254ea51381696b9927f7c62dd36e10ef00fcac
Summary:
Pull Request resolved: https://github.com/facebook/flipper/pull/1052
Fixed tests failing on Windows because jest auto-mocks works differently there
Reviewed By: passy
Differential Revision: D21237618
fbshipit-source-id: 31c7e92b7f8ae84c1e65fd37428204452b3f1b00
Summary: Added eslint rule "no-extraneous-imports" which disallow using modules which are not listed as dependencies in the corresponding package.json. Fixed a bunch of reported errors after the rule applied.
Reviewed By: passy
Differential Revision: D21186848
fbshipit-source-id: 0af9ac4b3fffdfd0ab7c23ae4ff12a3f5989d5e9
Summary: This shaves another ~6MB of the zipped build. Less than I would have thought, but I haven't looked at the alternative options so far at all.
Reviewed By: nikoant
Differential Revision: D21131772
fbshipit-source-id: 641515d6f36fdeeffa5be81d64c2c1043a9e4e34
Summary:
Added versioning for plugin format.
The first version is where "main" points to source code entry and plugins are bundled by Flipper in run-time on loading them.
The second version is where "main" points to the already existing bundle and Flipper just loads it without bundling. The plugins of version 2 must be bundled using "flipper-pkg" tool before publishing.
Changelog: Support new packaging format for plugins.
Reviewed By: mweststrate
Differential Revision: D21074173
fbshipit-source-id: 7b70250e48e5bd5d359c96149fb5b14e67783c4d
Summary: "flipper-pkg" added ~2MB to Flipper disttributive size, because of heavy dependencies which are only required for CLI functionality. See size warning in diff D21068373 in this stack where I added pkg as dependency to flipper. Here I'm splitting it into library and CLI packages, so Flipper app will only reference the library.
Reviewed By: passy
Differential Revision: D21087336
fbshipit-source-id: d9d62f1e75a835d1c0fa78ff1addb0d9a761a9c7
Summary: Moved getWatchFolders script to flipper-pkg and used it from other packages. Script helps to resolve all the folders with package dependencies including symlinked folders in case if plugin is a part of yarn workspaces.
Reviewed By: mweststrate
Differential Revision: D21068373
fbshipit-source-id: 8691837fdb1aef333dab4c13d8758262838d36ee
Summary:
Pull Request resolved: https://github.com/facebook/flipper/pull/998
After this diff all the default plugins (which are distributed with Flipper) will be included into the main app bundle instead of bundling each of them separately and then loading from file system. This is done by auto-generating plugins index in build-time and importing it from Flipper app bundle, so Metro can follow these imports and bundle all the plugins to the app bundle.
This provides several benefits:
1) reduced Flipper bundle size (~10% reduction of zipped Flipper archive), because Metro bundles each of re-used dependencies only once instead of bundling them for each plugin where such dependency used.
2) Faster Flipper startup because of reduced bundle and the fact that we don't need to load each plugin bundle from disk - just need to load the single bundle where everything is already included.
3) Metro dev server for plugins works in the same way as for Flipper app itself, e.g. simple refresh automatically recompiles bundled plugins too if there are changes. This also potentially should allow us to enable "fast refresh" for quicker iterations while developing plugins.
4) Faster build ("yarn build --mac" is 2 times faster on my machine after this change)
Potential downsides:
1) Currently all the plugins are identically loaded from disk. After this change some of plugins will be bundled, and some of them (third-party) will be loaded from disk.
2) In future when it will be possible to publish new versions of default plugins separately, installing new version of such plugin (e.g. with some urgent fix) will mean the "default" pre-built version will still be bundled (we cannot "unbundle" it :)), but we'll skip it and instead load new version from disk.
Changelog: Internals: include default plugins into the main bundle instead producing separate bundles for them.
Reviewed By: passy
Differential Revision: D20864002
fbshipit-source-id: 2968f3b786cdd1767d6223996090143d03894b92
Summary:
Refactored plugins compilation script by splitting it into 3 scripts: 1) search plugins 2) compile plugins 3) watch plugins.
No functional changes in this review, it's just preparation for further changes.
Reviewed By: passy
Differential Revision: D20860443
fbshipit-source-id: 02900430199875574b992e597d09b82d0d7f32ef
Summary: We cannot just add internal plugins as workspaces to the root package.json in "sonar/desktop" as they are not open-sourced, so public build will break. Instead, I have created root package.json for internal plugins and added all internal plugins as workspaces there. This means all these plugins will use the single root yarn.lock and installation of their dependencies will be faster. This also means that plugins can declare dependencies to other local packages included into workspaces and they will be symlinked automatically.
Reviewed By: mweststrate
Differential Revision: D20806237
fbshipit-source-id: f8b3327166963dec7da8ac74079682aebe4527e1
Summary:
Enabling Metro cache for dev mode. For release builds we reset the cache.
Cache is used for faster compilation in dev mode for both main and renderer bundles, as well as for plugins.
Currently we have few side effects based on env vars, so cache is invalidated when they are changed. Also the cache is invalidated when transformations are changed (changed code, bumped dependency etc). Also added a script to reset the cache if something is going wrong.
Reviewed By: mweststrate
Differential Revision: D20691464
fbshipit-source-id: 478947d438bd3090f052dbfa6ad5c649523ecacb
Summary:
SORRY FOR BIG DIFF, but it's really hard to split it as all these changes are cross-dependent and should be made at once:
1. Moved transformations to separate package "flipper-babel-transformer" and linked it using yarn workspaces to "static" and "pkg" packages where they are re-used. Removed double copies of transformations we had before int these two packages.
2. Converted transformations to typescript
3. Refactored transformations to avoid relying on file system paths for customisation (FB stubs and Electron stubs for headless build)
4. As babel transformations must be built before other builds - enabled incremental build for them and changed scripts to invoke the transformations build before other build scripts
5. As we need to deploy all the dependencies including the fresh "flipper-babel-transformer" as a part of "static" - implemented script which copies package with all the dependencies taking in account yarn workspaces (hoisting and symlinks)
Reviewed By: passy, mweststrate
Differential Revision: D20690662
fbshipit-source-id: 38a275b60d3c91e01ec21d1dbd72d03c05cfac0b
Summary:
Quick notes:
- This looks worse than it is. It adds mandatory parentheses to single argument lambdas. Lots of outrage on Twitter about it, personally I'm {emoji:1f937_200d_2642} about it.
- Space before function, e.g. `a = function ()` is now enforced. I like this because both were fine before.
- I added `eslint-config-prettier` to the config because otherwise a ton of rules conflict with eslint itself.
Close https://github.com/facebook/flipper/pull/915
Reviewed By: jknoxville
Differential Revision: D20594929
fbshipit-source-id: ca1c65376b90e009550dd6d1f4e0831d32cbff03
Summary:
1) moved "sonar/desktop/src" to "sonar/desktop/app/src", so "app" is now a separate package containing the core Flipper app code
2) Configured yarn workspaces with the root in "sonar/desktop": app, static, pkg, doctor, headless-tests. Plugins are not included for now, I plan to do this later.
Reviewed By: jknoxville
Differential Revision: D20535782
fbshipit-source-id: 600b2301960f37c7d72166e0d04eba462bec9fc1
Summary: This diff allows direct referencing pkg and doctor packages from the main bundle so they can be changed and used in the main app without publishing to npm
Reviewed By: mweststrate
Differential Revision: D20390416
fbshipit-source-id: af625a8b7cead4481743a88aec198493d363ce76
Summary:
Pull Request resolved: https://github.com/facebook/flipper/pull/878
1) Enabled eslint error reporting for unresolved imports and fixed all the errors
2) Enabled eslint for typings (d.ts) and fixed all the errors
Reviewed By: passy
Differential Revision: D20335151
fbshipit-source-id: 7b142281a406b32df0f02a5cd0d7d05eba941acd
