Summary:
This diff addresses two problems:
1. Since clients plugins can be active beyond having a connection, we have to make it possible for plugin authors to check if they are connected before they make a call.
2. if there is a custom `exportPersistedState`, plugins should be able to skip making calls if the device has disconnected.
Introducing this change makes it possible to interact with a reasonable level with disconnected clients, and makes it possible to create Flipper traces for disconnected clients.
Note that both items were already problems before supporting offline clients; as there can be a noticeable delay between disconnecting and Flipper detecting that (i've seen up to 30 secs). What happend previously in those cases is that the export would simply hang, as would other user interactions, as loosing the connection in the middle of a process would cause the promise chains to be neither rejected or resolved, which is pretty iffy.
Before this diff, trying to export a disconnected device would hang forever like:
{F369600601}
Reviewed By: nikoant
Differential Revision: D26250895
fbshipit-source-id: 177624a116883c3cba14390cd0fe164e243bb97c
Summary:
This diff introduces support for keeping clients around after they have disconnected. This is a pretty important debugging improvement, that will allow inspecting a device / app after it crashed for example.
With this diff, the current client is just kept around until it connects again, instead of throwing clients immediately away if they disconnect. After this change, ArchivedClients will only be created by imports / exports, and no longer by disconnects. Initially I played with improving the creation of archived devices, by migrating all plugin state over from the original client to the archive, but I discovered that is very prone, as it would be a lot of pointer redistribution (plugins would point to a different client / device etc). While in contrast, disconnected clients is already an existing concept in Flipper, so reusing that keeps all the changes relatively simple.
Note that we could potentially still reuse old clients around after reconnected, but it would become much harder to reason about how plugins would behave if they missed updates for a while, so throwing away the device / clients and starting with a fresh slate sounds safer. So I figured that chance to be too risky for now, but would probably be good follow up work.
Issues with import / export, UX, and making calls to to a disconnected client will be addressed in follow up diffs
Changelog: Clients will retain their state after being disconnected, until they reconnect again
Reviewed By: nikoant
Differential Revision: D26224677
fbshipit-source-id: feb9d241df2304341c2847fe7fd751ac54c045f6
Summary:
This removes the Non-Sandy UI from the Flipper codebase. It is a pretty rough scan for unused components, over time when converting more advanced components to Ant design probably even more code can be removed.
Partially used `npx ts-purge` to reveal never imported source files.
Changelog: It is no longer possible to opt out of the new Sandy UI
Reviewed By: jknoxville
Differential Revision: D25825282
fbshipit-source-id: 9041dbc7e03bce0760c9a0a34f1877851b5f06cf
Summary: fb/App didn't really have a difference with fb-stubs/App, except that one checks for employee presence. However, since that is already controlled by config.checkFbEmployee, de-duped this to a single `<AppWrapper>` component, making the code a bit more easy to follow
Reviewed By: priteshrnandgaonkar
Differential Revision: D25824521
fbshipit-source-id: 8e16f0b29ec5d12475eaf14acd9dbc7df91191a2
Summary: Flipper receives a lot of messages which cannot be handled when plugins unbundled. Showing them as warnings is un-actionable with such big amount, so I disabled them.
Reviewed By: mweststrate
Differential Revision: D25557790
fbshipit-source-id: 1a0bc3df2848b194b72ea6d775ffabe416672fee
Summary:
This diff will enabling freezing (making immutable) of all data we receive from the device. This prevents dev mistakes causing components not to update or logic being hard to reason about. Also this makes Immer's `produce` faster in the typical case
Since this is potentially a risky change, that might break existing plugin logic, it has been put behind a GK https://www.internalfb.com/intern/gatekeeper/projects/flipper_frozen_data/
I did some quick exploratory testing on all plugins available for Facebook iOS / Android, and the only plugin that caused trouble was Fresco, which is fixed in this diff as well.
Reviewed By: nikoant
Differential Revision: D25055056
fbshipit-source-id: 8525511f4a8a0221740a6e1371ce7f2b757a203e
Summary: `unstablebatched_updates` should be used whenever a non-react originating event might affect multiple components, to make sure that React batches them optimally. Applied it to the most import events that handle incoming device events
Reviewed By: nikoant
Differential Revision: D25052937
fbshipit-source-id: b2c783fb9c43be371553db39969280f9d7c3e260
Summary: This prefixes APIs of `flipper-plugin`, that are used by Flipper, but should not be used by plugins directly, with `_`. Also added tests to make sure we are always intentional when extending the exposed APIs
Reviewed By: passy
Differential Revision: D24991700
fbshipit-source-id: ed3700efa188fca7f5a14d5c68250598cf011e42
Summary:
Navigation plugin is a special cause that will remain connected and process messages directly even when disabled, this is to make sure the bookmarks feature keeps working even when the plugin is not enabled.
Changelog: [Sandy][Navigation] on Android, the currently active deeplink of the application will now be shown in the sidebar
Reviewed By: jknoxville
Differential Revision: D24890375
fbshipit-source-id: eb5e4141740e0436396cea5a7aae24337f2e903e
Summary:
devices not always being readily available is causes a lot of complication in the api,
figured to resolve devices first before construction clients,
since clients not attached to a device are shown uncategorized anyway, making them practically un-interactable.
For more background info, see following chat.
{F344388883}
This diff will make it possible to only expose a synchronous api in Sandy
n.b. didn't update Navigation plugin, as that is done in a next diff
Reviewed By: jknoxville
Differential Revision: D24858332
fbshipit-source-id: 8339f831fbbc9c219add56a199364fde67adafc7
Summary: Implemented a way for re-loading single plugin on auto-update. This make it possible to apply update without full Flipper restart.
Reviewed By: mweststrate
Differential Revision: D23729972
fbshipit-source-id: ed30f7cde5a0537945db0b5bb6969ae8fde42cb6
Summary:
[interesting] since it shows how Flipper APIs are exposed through sandy. However, the next diff is a much simpler example of that
This diff adds support for adding menu entries for sandy plugin (renamed keyboard actions to menus, as it always creates a menu entry, but not necessarily a keyboard shortcut)
```
client.addMenuEntry(
// custom entry
{
label: 'Reset Selection',
topLevelMenu: 'Edit',
handler: () => {
selectedID.set(null);
},
},
// based on built-in action (sets standard label, shortcut)
{
action: 'createPaste',
handler: () => {
console.log('creating paste');
},
},
);
```
Most of this diff is introducing the concept of FlipperUtils, a set of static Flipper methods (not related to a device or client) that can be used from Sandy. This will for example be used to implement things as `createPaste` as well
Reviewed By: nikoant
Differential Revision: D22766990
fbshipit-source-id: ce90af3b700e6c3d9a779a3bab4673ba356f3933
Summary:
This diff makes sure that devices will actually instantiate applicable sandy device plugins. Similar to how client plugins are owned by Client, device plugins are directly owned by BaseDevice, which significantly simplifies life cycle management and doesn't dispatch updates to all Redux connect components whenever something irrelevant changes.
Also made sure `device.teardown()` is called. That API already existed, but wasn't used or implemented before.
Updated Flipper test utils to support testing device plugins as well (both Sandy and classic ones)
Reviewed By: passy, nikoant
Differential Revision: D22693929
fbshipit-source-id: 73b2b8666ef7a0e748ea89360db84734d37eb5be
Summary:
let's finally inspect flipper with flipper!
Here we have:
1) a self inspection client which implements FlipperClient interface from js sdk and FlipperClientConnection. It links back and front parts of self inspection
2) simple plugin (UI) to show messages
3) back part of that plugin - it sends all received messages to UI part via client
4) we initialize self inspection for dev builds only
P. S. filesystem dependency will be replaced with npm one before I ship it (need to publish to npm first)
Reviewed By: mweststrate
Differential Revision: D22524533
fbshipit-source-id: 5c77e2f7b50e24ff7314e791a4dfe3c349dccdee
Summary:
Made Sandy plugins part of the export / import logic
Note that in the export format there is now a `pluginStates2` field. It could have been put in `pluginStates` as well, but I started with that and it felt very weird, as the `pluginStates` are supposed to reflect what is living in the Redux stores, where sandy state doesn't live in the Redux store (not directly at least). Trying to store it in the same field made things overcomplicated and confusing.
Reviewed By: jknoxville
Differential Revision: D22432769
fbshipit-source-id: d9858e561fad4fadbf0f90858874ed444e3a836c
Summary:
While writing unit tests discovered a bug that disabling a plugin doesn't guarantee cleaning the messagequeues (both the buffer in client and the messagequeue reducer).
Fixed that. That was thanks to @#%@#$@#%@ Redux a lot harder than it should be; as 'STAR_PLUGIN' reasons about a plugin + app name, while the message queue reducer would need to deduct the plugin keys from that, but it can't because that mapping is stored in the connections reducers. So I moved the `STAR_PLUGIN` action handling to the root reducer, sot that it can reason about the state of multiple reducers, which looked like the least of all evils. For more ranting about that and possible alternative solutions: https://twitter.com/mweststrate/status/1277556309706117122
Reviewed By: nikoant
Differential Revision: D22284043
fbshipit-source-id: 35d0a8ba3a21a5959d2bb6ef17da3ff5077f48fd
Summary: This diffs adds the capability to listen to messages in Sandy plugins. Although API wise it looks more like the old `this.subscribe`, semantically it behaves like the `persistedStateReducer`; messages are queued if the plugin is enabled but not active.
Reviewed By: nikoant
Differential Revision: D22282711
fbshipit-source-id: 885faa702fe779ac8d593c1d224b2be13e688d47
Summary:
This sets up the initial infra that is to be used by plugin devs to test plugins.
There is not much yet to see, as there is no state or message sending yet. But at least the life cycle of plugins can be test, things are strongly typed and everything is in the place where it should be :)
N.b. the import difference with these utils and the createFlipperMock utilities in Flipper are
1. this testing infra is entirely inside flipper-plugin package, so that plugin devs don't need flipper as a dependency
2. this testing infra doesn't provide abstractions for plugin / device / client switching; it tests plugins purely in isolation of the rest of the world (except for firing `onConnect` / `onDisconnect` which is normally the effect of switching plugins)
Reviewed By: nikoant
Differential Revision: D22255262
fbshipit-source-id: b94ccbab720d2b49428a646aed3c55af71a5bc80
Summary:
Introduced hooks that are called whenever the plugin is connected / disconnected to it's counter part on the device.
There is some logic duplication between `PluginContainer` for old plugins, and `PluginRenderer` for new plugins, mostly caused by the fact that those lifecycles are triggered from the UI rather than from the reducers, but I figured refactoring that to be too risky.
Reviewed By: jknoxville
Differential Revision: D22232337
fbshipit-source-id: a384c45731a4c8d9b8b532a83e2becf49ce807c2
Summary: Replaced `instanceof` checks with `isSandyPlugin` utility. That is cleaner to read and makes it easier to find places where we make exceptions for Sandy plugins
Reviewed By: jknoxville
Differential Revision: D22206707
fbshipit-source-id: b44a1b585424f3b9bf0d7ce200c34107f03ed55e
Summary:
This diff makes sure sandy plugins are initialized.
Sandy plugins are stored directly in the client for two reasons
1. we want to decouple any plugin state updates from our central redux store, as redux is particularly bad in handling high frequency updates.
2. The lifecycle of a plugin is now bound to the lifecycle of a client. This means that we don't need 'persistedStore' to make sure state is preserved; that is now the default. Switching plugins will no longer reinitialize them (but only run specific hooks, see later diffs).
3. PersistedState will be introduced for sandy plugins as well later, but primarily for import / export / debug reasons.
A significant difference with the current persistent state, is that if a client crashes and reconnects, plugins will loose their state. We can prevent this (again, since state persisting will be reintroduced), but I'm not sure we need that for the specific reconnect scenario. Because
1. we should fix reconnecting clients anyway, and from stats it looks to happen less already
2. reconnects are usually caused by plugins that aggregate a lot of data and get slower over time. Restoring old state also restores those unstabilites.
For the overview bringing back the archi picture of earlier diff:
{F241508042}
Also fixed a bug where enabling background plugins didn't enable them on all devices with that app.
Reviewed By: jknoxville
Differential Revision: D22186276
fbshipit-source-id: 3fd42b577f86920e5280aa8cce1a0bc4d5564ed9
Summary:
So far there were 2 types of plugins: `FlipperPlugin` and `FlipperDevicePlugin`. This introduces a third kind: `SandyPluginDefinition`.
Unlike with the old plugins, the export of the module is not directly exposed as the plugin definition. Rather, we use class `SandyPluginDefinition` (instance) that holds a loaded definition and its meta data separately (`PluginDetails`). This means that we don't have to mix in and mutate loaded definitions, and that for unit tests we can avoid needing to provide a bunch of meta data. This also prevents a bunch of meta data existing on two places: on the loaded classes as static fields, and in the meta data field of the loaded class as well. Finally, we can now freely extends the `PluginDetails` interface in flipper, without needing to store it on the loaded classes and we are sure that no naming conflicts are caused by this in the future.
For compatibility with the existing code base, common fields are delegated from the `SandyPluginDefinition` class to the meta data.
Also cleaned up types around plugins a little bit and removed some unnecessary casts.
For all features that reason about plugins in general (such as exports), sandy plugins are ignored for now.
`SandyPluginInstance` is worked out in further diffs
The `instanceof` calls are replaced by a utility function in later diffs.
{F241363645}
Reviewed By: jknoxville
Differential Revision: D22091432
fbshipit-source-id: 3aa6b12fda5925268913779f3c3c9e84494438f8
Summary:
See previous diff.
Achieves the same optimization as in the mentioned diff, but this time by only debouncing the messages as they arrive over the socket, and not the state updates caused by Redux directly. This means that plugin rendering won't be debounced anymore until we address this more fundamentally.
With this change there is a double level buffering:
1. A buffer that stores all incoming messages (that are not replies to requests)
2. A buffer that stores messages we are interested in in the plugin queue, unless the plugin is active (this we already had).
This still fixes the issue that too chatty plugins cause to many updates foreground plugin (the problem we tried to fix originally), without debouncing plugin rendering if it is needed to update for any other reason.
Another nice benefit is that previously every received message would trigger a store update in Redux which would cause all connected components to evaluate their subscriptions (and then bail out in the typical case). Now we will only update the redux store every 200 ms.
Changelog: Foreground plugins will burn less CPU when they're very chatty
Reviewed By: jknoxville
Differential Revision: D21858849
fbshipit-source-id: c72352e569a8a803bbedffb71b17b11fcefee043
Summary:
This diff fixes an issue where we don't want to have the Navigation plugin be disabled as background plugins like all other plugins, so that the breadcrumb navigation keeps working.
Yet another hack concerning the super useful Navigation plugin. On a positive note, since connection management for background is not entirely managed by the Desktop and not the native said, these kind of exceptions are fairly easy to make :)
Reviewed By: passy
Differential Revision: D21089537
fbshipit-source-id: 209954ff35c95e066fe688a60ad46ccfc3835c44
Summary:
Background for this diff: https://fb.quip.com/KqEfAlKYlgme
Some plugins don't respect that stuff (livefeed and graphql), but for others it seems to work fine.
This is just a PoC, there are some present bugs concerning the combination of selecting and bg plugins
Questions to investigate:
- [x] make sure that LiveFeed and GraphQL disconnect properly. There might be more plugins that need that
- [x] verifiy that we don't loose one of the original goals of background plugins, e.g. QPL collecting and sending data from device start. Does this still work as intended after this change?
- [x] how can we observe / measure improvements? Are dev builds more responsive after this? Is the layout inspector smoother for example because no QPL plugins are interweaved?
- [x] how is forward and backward compatibility?
- If Flipper is updated, but device not: No change I think, as getBackgroundPlugins() will return an empty set, and background plugins are initiated as usual, so old behavior
- If device is updated, but Flipper not, background plugins won't be started until they are selected. This is a degradation, but hopefully explainable.
- [x] Verify QPL buffer is not unbounded
- [x] Share architecutre changes with team
For Graphql updates: D20943455
Added runtime stats to monitor network traffic (sadly had to redo that since scuba couldn't handle the data format used at first, so probably will hold of landing this diff a week to make sure we can see some effects)
Follow up work:
[x] wait until we released the stat tracking before we release this, to be able to measure the effect?
[x] make sure graphql fix lands
[ ] use side effects abstraction
[ ] fix other background plugins (android only) or fix it in a generic way:
{F234394286}
Changelog: Background plugins will no longer receive a Flipper connection if they are disabled. This should significantly reduce the overall load of Flipper both on the device and desktop when unused plugins are disabled used, which could otherwise generate 10MB/s of network traffic certain scenarios. All plugins *should* be able to handle to this gracefully, but since this is quite a fundamental change, reach out to the Flipper team when in doubt!
Reviewed By: jknoxville
Differential Revision: D20942453
fbshipit-source-id: b699199cb95c1b3e4c36e026b6dfaee7d1652e1f
Summary: I think it is good to have error handlers where-ever possible, so that there is historical data in our monitoring when we need it, and can track from which point of the code it originates :)
Reviewed By: jknoxville
Differential Revision: D21040059
fbshipit-source-id: 1c07fbfa65379739554bc98f83761ae97870ba82
Summary:
Measure how many byte we receive per plugin, and add this to the plugin stats that are collected
Will add a graph to the flipper dashboard, and probably a small visualization in a next diff as well.
Reviewed By: priteshrnandgaonkar
Differential Revision: D20917583
fbshipit-source-id: bb341531ecf8492080af82c56e73c0ec608f7b36
Summary:
Before we dropped all messages if the client connected before the device has been registered. Which happens typically on iOS (it can actually take very long, will investigate that as well, but not in this diff).
However, we don't need to wait for the device to process messages, we just need its serial, which we already know (given that we use that query id to find the matching device in the first place).
I think this will greatly improved perceived stability for iOS
Reviewed By: jknoxville
Differential Revision: D20734892
fbshipit-source-id: f98e8d31558ef606b9a8287e03fc41ab6c3a087d
Summary: See previous two diffs, this applies the abstraction to our code base
Reviewed By: passy
Differential Revision: D20679687
fbshipit-source-id: 05e340dca3f832971783a844a78d1ffd553ff9d2
Summary:
Quick notes:
- This looks worse than it is. It adds mandatory parentheses to single argument lambdas. Lots of outrage on Twitter about it, personally I'm {emoji:1f937_200d_2642} about it.
- Space before function, e.g. `a = function ()` is now enforced. I like this because both were fine before.
- I added `eslint-config-prettier` to the config because otherwise a ton of rules conflict with eslint itself.
Close https://github.com/facebook/flipper/pull/915
Reviewed By: jknoxville
Differential Revision: D20594929
fbshipit-source-id: ca1c65376b90e009550dd6d1f4e0831d32cbff03
Summary:
Our usage of requestIdleCallback (probably) causes more trouble than it solves:
1. It makes sure everything is processed asynchronously. But since everything is arriving over a network stack, that is already the case without wrapping it again to run on a separate event loop tick
2. The timeout we set before `500` forces the app to give _more_ priority to message processing instead of less
3. In a next diff (D20151700) in this stack we will make sure that messages are not processed immediately, but simple stored, which should not be significantly more expensive (probably even cheaper) than scheduling another tick on the event loop
Reviewed By: jknoxville
Differential Revision: D20557104
fbshipit-source-id: 6cc10ba537e3cb5f31e6c32e1fdeb57c20f06f17
Summary: Did run for 2 months stable now, time to remove some dead code.
Reviewed By: passy
Differential Revision: D20556785
fbshipit-source-id: 514673995212d62d21744f304286caa6e91007fb
Summary:
1) moved "sonar/desktop/src" to "sonar/desktop/app/src", so "app" is now a separate package containing the core Flipper app code
2) Configured yarn workspaces with the root in "sonar/desktop": app, static, pkg, doctor, headless-tests. Plugins are not included for now, I plan to do this later.
Reviewed By: jknoxville
Differential Revision: D20535782
fbshipit-source-id: 600b2301960f37c7d72166e0d04eba462bec9fc1
