Summary:
flipper-server-companion depends on flipper-plugin. flipper-plugin includes dependencies that run only in a browser. Splitting flipper-plugin into core and browser packages helps to avoid including browser-only dependencies into flipper-server bundle.
As a result, bundle size could be cut in half. Subsequently, RSS usage drops as there is twice as less code to process for V8.
Note: it currently breaks external flipper-data-source package. It will be restored in subsequent diffs
Reviewed By: lblasa
Differential Revision: D38658285
fbshipit-source-id: 751b11fa9f3a2d938ce166687b8310ba8b059dee
Summary: Current implementation uses type `string` as a key for indexing items stored in datasource. However, users can provide any key as an index which means that the type of index item can be anything, not only string. This diff introduces a more refined types for the key. It adds another requirement to provide a key property to a generic which is used to infer the index type.
Reviewed By: mweststrate, aigoncharov
Differential Revision: D31895751
fbshipit-source-id: 19ba907bd6f35df87e3fa442db5fc5cec6af174d
Summary:
Per title.
Feature will be used in several plugins in next diffs.
Differential Revision: D29514456
fbshipit-source-id: c12427c2a7c53fa01cd1c7f429be8611be55496d
Summary:
Changelog: [Flipper] Improve serialisation mechanism format & speed
The default serialisation mechanism used by Flipper to serialise plugin states is very flexible, taking care of maps, sets, dates etc. However, it is also really slow, leading to issues like in the related tasks, and work arounds like D17402443 (98bc01618f) to skip the whole process for plugins.
This diff changes the serialisation mechanism to have a better trade off between speed and convenience: For now we will only apply the smart serialisation for objects living at the _root_ of the serialised object, but it won't be applied recursively.
This sounds like a dangerous change, but works well in practice:
* I went through all `persistedState` and `createState` definition (the types), and the idea that complex types like Map and Set only live at the root of the persisted state holds up nicely. That makes sense as well since plugins typically store literally the same data as that they have received over the wire, except that they put it in some maps, sets etc.
* I introduced `assertSerializable` that only runs in dev/test, which will check (recursively, but without all the cloning) to see if a tree is indeed serialisable.
* The fact that by swapping this mechanism rarely existing unit test for exportData needed changes proves that the assumption that only roots are relevant generally upholds (or that plugin authors don't write enough tests ;-)).
* I verified that popular plugins still import / export correctly (actually *more* plugins are exportable now than before, thanks to sandy wrapper introduced earlier)
Reviewed By: jknoxville
Differential Revision: D29327499
fbshipit-source-id: 0ff17d9c5eb68fccfc2937b634cfa8f4f924247d
Summary: some type simplifications, that makes it easier to reuse data sources and helps type inference
Reviewed By: passy
Differential Revision: D28413380
fbshipit-source-id: 261a8b981bf18a00edc3075926bd668322e1c37d
Summary:
Added a microbundle based build setup to the data-source folder to be able to package just that folder.
For simplicity / iteration speed, this is only used to publish externally. Our own code still references the source files directly.
More strict separation can be done later if there is external adoption.
Reviewed By: nikoant
Differential Revision: D28056699
fbshipit-source-id: a011b615cfffeff8ecb879bd7281a71085cea965
Summary:
To make the DataSource abstraction reusable for other teams and an upcoming talk, this diff moves all DataSource storage & virtualization logic in one folder.
Will set up a build process and demo project in later diffs.
Reviewed By: nikoant
Differential Revision: D28056700
fbshipit-source-id: 7cfe5b40bbbe387da711f765a604a45029d451c7
Summary:
converted the network plugin to use DataSource / DataTable. Restructured the storage to contain a single flat normalised object that will be much more efficient for rendering / filtering (as columns currently don't support nested keys yet, and lazy columns are a lot less flexible)
lint errors and further `flipper` package usages will be cleaned up in the next diff to make sure this diff doesn't become too large.
The rest of the plugin is converted in the next diff
Reviewed By: nikoant
Differential Revision: D27938581
fbshipit-source-id: 2e0e2ba75ef13d88304c6566d4519b121daa215b
Summary:
This help handle updates to objects containing JSONObject type (generates typescript errors around infinite recursion:
> Type instantiation is excessively deep and possibly infinite.ts(2589)
Reviewed By: mweststrate
Differential Revision: D28098502
fbshipit-source-id: a8e7f785b03bce7c8e1c0b355e2eca2e087b1881
Summary: This diff exposes the createTablePlugin from flipper-plugin, so that createTablePlugin based plugins can be converted to Sandy as well
Reviewed By: jknoxville
Differential Revision: D28031227
fbshipit-source-id: 8e9c82da08a83fddab740b46be9917b6a1023117
Summary: Allow subscribing to Atom state changes
Reviewed By: mweststrate
Differential Revision: D28027692
fbshipit-source-id: 24fd7ea16b013c364bbb1d25b30c48bc698db014
Summary: Introduce the DataList component. Not feature complete yet, but core functionality is present so that people can use it during the convertathon. It is used to implement the route list in the network mock dialog
Reviewed By: priteshrnandgaonkar
Differential Revision: D27046716
fbshipit-source-id: a247ce7032b350b31bf55962ca4268e30f43471a
Summary: made a typescript booboo, and broke the test thereby. Also reduced the test set to avoid OOMs
Reviewed By: passy
Differential Revision: D27261187
fbshipit-source-id: bcb61bf95419421d9c33e7e0a944717797f96dfe
Summary:
This diff is primarily cosmetic, just pushing code around to make the API more intuitive. Most importantly, DataSource was split into DataSource and DataSourceView classes, the latter being accessible through `datasource.view`.
The benefit of this is two fold:
1. Conceptually it is much clearer now which operations operate on the _source_ records, and which ones on the derived _view_.
2. This will make it easier in the future to support multiple views to be based on a single data source.
This refactoring also nicely found 2 cases where datasource logic and view logic were mixed.
The only semantic change in this diff is that both DataSource and DataSourceView are now iterable, so that one can do a `for (const record of ds)` / `for (const record of ds.view)`
Reviewed By: nikoant
Differential Revision: D26976838
fbshipit-source-id: 3726e92b3c6ee3417dc66cbbe6e288797eecf70e
Summary: Make sure that DataSources can be serialized directly with a single setting, just like plain state atoms
Reviewed By: nikoant
Differential Revision: D26944954
fbshipit-source-id: 2b0d625d7d67f27a7c2e33dd7c4b534dfa4d3e82
Summary:
Noticed in the previous diff that shift is relatively slow for sorted datasources, the reason is that it needs to do a lot of binary searches, and binary search / sorting a full data set is roughly ~20 times slower than resorting a full set, and we're dropping 10% of the data in our test. So if we are shifting too many items in a sorted set, we instead fall back to a rebuild (for non-sorted, shift is super fast because we only drop a bunch of items from the start).
Also solved some more perf related todo's, or made notes about them.
Reviewed By: nikoant
Differential Revision: D26913144
fbshipit-source-id: ee1c04fda1730653affdede0ad22da795e19c2af
Summary:
Added some performance tests for DataSource. Currently simply using jest to run them in a single run, so that is not the most isolated setup (we do GC between tests), but helps to find some global trends at least.
For every scenario two datasets are used, one of 100.000 items, and one of 200.000 items, to verify that all important functions scale roughly linearly or better.
The `append` and `update` test cases perform 1000 insertions / updates. All other tests are singular.
The keyed vs unkeyed variation verifies that we don't drop performance if we maintain a by-key lookup table.
The sorted variations start with an initially already sorted and filtered setup. This nicely show that the datasource really starts to shine with its insertion sort versus full reallocating and sorting
The reference fake implementation does what we do in most cases in Flipper: shallow clone and allocate an entirely new array to append / update data to preserve immutability. Its comparison is pretty terribly, especially considering that in the perf tests we 'render' only once, skewing the stats in favor of the fake implementation: only at the end of the entire batch of updates we sort & filter once (so after inserting a thousand items for example).
In contrast the datasource tests will keep its data sorted at all times, so 'rendering' is already included in the measurements. For the fake datasource, resorting the full 200K rows after each insert would pretty much put bitcoin caused global warming to shame. Also note that the increased GC pressure isn't incorporated in the fake implementation, as we GC outside the measurements.
Reviewed By: nikoant
Differential Revision: D26913145
fbshipit-source-id: 955f1923dce40997cd2e81ea9e80832c6e71c99c
Summary: This diff implements the shift operation, which removes (efficiently) the first (oldest) N records from the datasource. Also implemented a `limit` option to truncate automatically and limit memory usage
Reviewed By: nikoant
Differential Revision: D26883673
fbshipit-source-id: c5ebaf2a327d56cbbe38280c6376c833bcf68b8c
Summary: Implemented `remove`, which, for a typical data source should not be needed. But that would be famous last words and wanted to prevent painting ourselves in a corner, so implemented it. Also because part of the logic is need for the `shift` operation (see next diff), which is much more important.
Reviewed By: priteshrnandgaonkar
Differential Revision: D26883672
fbshipit-source-id: 0dbfcdd3d5a16c4a2d53b0272000d183c67d0034
Summary: ..and some earlier reviews comments has been processed + some fine tuning on the ui
Reviewed By: priteshrnandgaonkar
Differential Revision: D26816559
fbshipit-source-id: adf2586763be185ee8e7cc22b2827ecefe4e4cab
Summary:
This diff implements the remaining features in the logs plugin:
- deeplinking
- merging duplicate rows
The logs plugin source code has now been reduced from originally `935` to `285` LoC. All optimisation code has been removed from the plugin:
* debouncing data processing
* pre-rendering (and storing!) all rows
Finally applied some further styling tweaks and applied some renames to DataTable / DataSource + types finetuning. Some more will follow.
Fixed a emotion warning in unit tests which was pretty annoying.
Reviewed By: passy
Differential Revision: D26666190
fbshipit-source-id: e45e289b4422ebeb46cad927cfc0cfcc9566834f
Summary:
Added styling / coloring to the new logs plugin, to bring it closer to feature completeness. Made the colum headers slightly more compact
Also made the API more foolproof by introducing the `useAssertStableRef` hook, that will protect against accidentally passing in props that would invalidate rendering every time.
Reviewed By: passy
Differential Revision: D26635063
fbshipit-source-id: 60b2af8db3cc3c12d8d25d922cf1735aed91dd2c
Summary:
This diff has some jak-shaving UX improvements after playing with the DataTable a bit more:
Selection
* deselecting a row from a larger set will make the last selected item the default selection
* re-selecting an item in a single selection will unselect it
Column Filtering
* Introduced button to toggle between filtering on all, nothing, and the values present in the current selection
Column sorting
* The up / down arrows are now inidividually clickable, rather than action as a general toggle
* Title still works as a general toggle between asc / desc / not sorted
Context menu
* I found the context menu for column selection and on the selected rows itself a bit finicky to find and click and not super intuitive for noob users. Merged both menus instead into a single hamburger menu adjacent to the search bar
Reviewed By: passy
Differential Revision: D26580038
fbshipit-source-id: 220f501a1d996acbd51088c08ea866caed768572
Summary: Show a hint during scrolling of the relative offset the user is looking at. This is based on the current virtualisation window, so not 100% accurate, but probably still provides the right signal to the user. See the bottom right of the recording
Reviewed By: nikoant
Differential Revision: D26450261
fbshipit-source-id: 206a860024e346c6b872edc3fc7919019046a6d7
Summary:
First rudementary setup of DataTable component that follows a data source. Initially used react-virtuose library, but it performed really badly by doing expensive layout shifts and having troublesome scroll handling. Switched to react-virtual library, which is a bit more level, but much more efficient, and the source code is actually understandable :)
Features:
- hook up to window events of datasource
- high and low prio rendering, based on where the change is happening (should be optimized further)
- sticky scrolling support
- initial column configuration (custom rendering, styling, columns etc will follow in next diffs)
Reviewed By: nikoant
Differential Revision: D26175665
fbshipit-source-id: 224be13b1b32d35e7e01c1dc4198811e2af31102
Summary: This diff introduces the possibility to subscribe to the `output` set of the datasource. It emits three possible event: `reset`, `update`, `shift`.
Reviewed By: jknoxville
Differential Revision: D26100104
fbshipit-source-id: b5fac2289206fab9fb8a437b96ab84034a8b5832
Summary:
Context: https://fb.workplace.com/notes/470523670998369
Per title, also restructured internal logic so that only on intermediate data structure is needed: `output`. This is because we don't need store filter information because we can run that plainly, on the incoming events without storing anything. Sorting is performed directly on `output`. Reverse isn't performed at all, but rather applied lazily when requesting a specific item (so it isn't reflected in output)
Reviewed By: nikoant
Differential Revision: D25976469
fbshipit-source-id: 777f8fdeba09729e19c97c176525b702066b6c2e
Summary:
clear will drop all current records but keep any view preferences. This typically relates to the "clear" button in Flipper.
reset on the other hand will keep the current records, but just reset the view preferences to the default, dropping any filters and sorting criteria
Reviewed By: nikoant
Differential Revision: D25975612
fbshipit-source-id: 5b419f259bffc049daf125090c6754aa6528919b
Summary:
For context see https://fb.workplace.com/notes/470523670998369
This diff adds support for reversing the data collection (in a table this would be used to toggle between ascending and descending sorting). The actual implementation is cleaned up in next diffs and the intermediate collection introduced here is dropped, so this diff is basically only about the unit tests, the implementation is not interesting at this point.
Reviewed By: nikoant
Differential Revision: D25975353
fbshipit-source-id: 2da6da2ed940c2e49e1986696d9b93a7b984db9b
Summary:
For context see https://fb.workplace.com/notes/470523670998369
This diff adds the capability to apply a sorting, and inserts item in a sorted way using binary search in a temporarily intermediate collection. (That collection is optimized away in later diffs, so it is mostly the idea and the tests that are interesting)
Reviewed By: nikoant
Differential Revision: D25953336
fbshipit-source-id: a51b05e25242f0835280ada99798676311511ef0
Summary:
For context see https://fb.workplace.com/notes/470523670998369
This diff introduces the DataSource abstraction, that can store records. If a key is set a key -> record mapping is stored, to make it easy to update existing records using `upsert`, without knowing their exact index.
Internal storage will be slightly altered in upcoming diffs, so don't pay to much attention to that part.
Reviewed By: nikoant
Differential Revision: D25953337
fbshipit-source-id: 1c3b53a2fcf61abaf061946be4af21d2aecc6c6d
Summary:
This diff stack introduces support for keeping devices and clients around after they have disconnected. This is a pretty important debugging improvement, that will allow inspecting a device / app after it crashed for example.
This feature existed partially before, but only supported Android, and only support plugins with persisted state; as it replace the current device with an archived version of the same device. In practice this didn't work really well, as most plugins would not be available, and all non-persisted state would be lost.
This diff makes sure we can keep devices around after disconnecting, the next one will keep the clients around as well. And explain some code choices in more detail.
Note that `Device.isArchived` was an overloaded term before, and even more now (both representing imported and disconnected devices), will address that in a later diff.
https://github.com/facebook/flipper/issues/1460https://github.com/facebook/flipper/issues/812https://github.com/facebook/flipper/issues/1487
Changelog: iOS and Android devices will preserve their state after being disconnected
Reviewed By: nikoant
Differential Revision: D26224310
fbshipit-source-id: 7dfc93c2a109a51c2880ec212a00463bc8d32041
Summary: Per title, this allows for pre-processing data after it is deserialized and before it is stored in the plugin
Reviewed By: nikoant
Differential Revision: D26126423
fbshipit-source-id: bc08a6ab205d2a0d551515563cd85a197595ddb2
Summary: `unstablebatched_updates` should be used whenever a non-react originating event might affect multiple components, to make sure that React batches them optimally. Applied it to the most import events that handle incoming device events
Reviewed By: nikoant
Differential Revision: D25052937
fbshipit-source-id: b2c783fb9c43be371553db39969280f9d7c3e260
Summary:
Converted the Navigation plugin to Sandy, and updated Locations bookmark accordingly.
This is a prerequisite step of supporting the bookmarkswidgetin the new AppInspect tab.
Updated LocationsButton accordingly, and overal simplified implementation a bit; locationsbutton now reuses the logic of the NavigationPlugin, rather than reimplemting it. This reduces code duplication and also makes sure the state between plugin and location button stays in sync.
Made sure that search providers are derived and cached rather than stored, again simplifying logic
That being said, the navigation plugin is buggy, but all these things failed before this diff as well:
* No events happening when using iOS, despite the plugin being enabled. But these seems to be a long time know issue, looks like it was never implemented
* Not sure if the parameterized bookmarks is working correctly
* screenshots not always happening at the right time (but fixed a race condition where the wrong bookmark might get updated)
* Locations button doesn't show up if the navigation plugin is supported but not enabled (will try to fix in next diff)
Would be great if bnelo12 could do some exploratory testing to verify what ought to be working, but currently isn't.
Reviewed By: cekkaewnumchai
Differential Revision: D24860757
fbshipit-source-id: e4b56072de8c42af2ada0f5bb022cb9f8c04bb47
Summary:
Converted the DeviceLogs plugin to sandy.
Kept logic and UI the same (so same batching, localstorage mechanisms etc). But used sandy api's for log subscribing, state, and separating the logical part of the component from the UI.
Note that some mechanisms work slightly different, like deeplinking and scrollToBottom handling, to reflect the fact that plugins are now long lived
Reviewed By: jknoxville
Differential Revision: D22845466
fbshipit-source-id: 7c98b2ddd9121dc730768ee1bece7e71bb5bec16
Summary: Introducing a base abstract class (blegh) to share some life cycle management between Device- and normal plugins. Cleaned up the test utils a bit as well + some old TODO's that now have been taken care of
Reviewed By: nikoant
Differential Revision: D22727089
fbshipit-source-id: 507816f1bfdbc6e7e71d4bd365b881b6710ca917
Summary:
This stack introduces Sandy device plugins, they are quite similar to normal plugins, but, a devicePlugin module is organized as
```
export function supportsDevice(device): boolean
export function devicePlugin(devicePluginClient)
export function Component
```
Device plugins get access to the device meta data and can subscribe to the `onLogEntry` callback and `onDestroy` lifecycle.
They will be able to store state just as normal plugins, but can't send or receive methods, so devicePluginClient is a bit limited.
This diff only sets up most of the new data structures, and makes sure everything still compiles and no existing tests fail.
To prevent this diff from becoming to big, actually loading, rendering and testing device plugins will be done in next diffs
Please take a critical look at the api proposed and the (especially) the public names used :)
Reviewed By: passy, nikoant
Differential Revision: D22691351
fbshipit-source-id: bdbbd7f86d14b646fc9a693ad19f33583a76f26d
Summary:
This plugin adds serialization capabilities to Sandy plugins buy setting the a `persist: <key>` flag. This shouldn't be used for state that is unserializable, too big, too sensitive, or irrelevant during export / import.
Using an explicit `persist` flag is done to make plugins robust to changes over time; as long as the key is kept the same, state variables can be renamed and reordered without breaking the import / export format. Also it allows us to detect some changes in the import / export format and warn about it.
Alternative designs considered but not implemented would be:
1. requiring the user to explicitly return the state from the factory (e.g. `const todos = createState([]); return { todos }`,
2. or construct the state from client (e.g. `const todos = client.createState([])`)
3. enable persistence by default, and store states in the order the states were created (much like useState in React). This was implemented in the first versions of this diff, but as pointed out in the discussions, this is too sensitive too (accidental) format changes, as the storage format would be quite implicit
A nice benefit of the current approach, especially compared with alternative approach 1, is that state being restored is immediately visible in the plugin factory. In other words, directly after initialization `const todos = createState([])`, the `todos.get()` is actually set to the state that is being restored, rather than having still the initial state which is only overridden rather. So this behaves very much like the `useState` hook in React.
Furthermore, in the future we could use the same `persist` key in combination with other options, such as `saveToLocalStorage`, in case some state acts as a 'preference' (T69989583).
`TestUtils.startPlugin` supports starting plugins with an initial state by using the optional `initialState` field
Actually wiring up the serialization and deserialization into the export / import functionality of Flipper is done in the next diff.
Reviewed By: jknoxville
Differential Revision: D22432770
fbshipit-source-id: 9a4849582c2f6f54d1e40f65a6cba73092c28fe8
Summary:
Introduced a minimal state abstraction so that state can be maintained in the plugin instance, but subscribe to by the UI.
At a later point we could pick an off the shelve solution like Recoil or MobX, or introduce cursors to read something deep etc etc, but for now that doesn't seem to be needed at all, and I think this will be pretty comprehensible for plugin authors (see also the 25th diff in this stack).
The api
```
createState(initialValue): Atom
Atom {
get() // returns current value
set(newValue) // sets a new value
update(draft => { }) // updates a complex value using Immer behind the scenes
}
// hook, subscribes to the updates of the Atom and always returns the current value
useValue(atom)
```
Reviewed By: nikoant
Differential Revision: D22306673
fbshipit-source-id: c49f5af85ae9929187e4d8a051311a07c1b88eb5
