Summary: Make sure that DataSources can be serialized directly with a single setting, just like plain state atoms
Reviewed By: nikoant
Differential Revision: D26944954
fbshipit-source-id: 2b0d625d7d67f27a7c2e33dd7c4b534dfa4d3e82
Summary:
Noticed in the previous diff that shift is relatively slow for sorted datasources, the reason is that it needs to do a lot of binary searches, and binary search / sorting a full data set is roughly ~20 times slower than resorting a full set, and we're dropping 10% of the data in our test. So if we are shifting too many items in a sorted set, we instead fall back to a rebuild (for non-sorted, shift is super fast because we only drop a bunch of items from the start).
Also solved some more perf related todo's, or made notes about them.
Reviewed By: nikoant
Differential Revision: D26913144
fbshipit-source-id: ee1c04fda1730653affdede0ad22da795e19c2af
Summary:
Added some performance tests for DataSource. Currently simply using jest to run them in a single run, so that is not the most isolated setup (we do GC between tests), but helps to find some global trends at least.
For every scenario two datasets are used, one of 100.000 items, and one of 200.000 items, to verify that all important functions scale roughly linearly or better.
The `append` and `update` test cases perform 1000 insertions / updates. All other tests are singular.
The keyed vs unkeyed variation verifies that we don't drop performance if we maintain a by-key lookup table.
The sorted variations start with an initially already sorted and filtered setup. This nicely show that the datasource really starts to shine with its insertion sort versus full reallocating and sorting
The reference fake implementation does what we do in most cases in Flipper: shallow clone and allocate an entirely new array to append / update data to preserve immutability. Its comparison is pretty terribly, especially considering that in the perf tests we 'render' only once, skewing the stats in favor of the fake implementation: only at the end of the entire batch of updates we sort & filter once (so after inserting a thousand items for example).
In contrast the datasource tests will keep its data sorted at all times, so 'rendering' is already included in the measurements. For the fake datasource, resorting the full 200K rows after each insert would pretty much put bitcoin caused global warming to shame. Also note that the increased GC pressure isn't incorporated in the fake implementation, as we GC outside the measurements.
Reviewed By: nikoant
Differential Revision: D26913145
fbshipit-source-id: 955f1923dce40997cd2e81ea9e80832c6e71c99c
Summary: This diff implements the shift operation, which removes (efficiently) the first (oldest) N records from the datasource. Also implemented a `limit` option to truncate automatically and limit memory usage
Reviewed By: nikoant
Differential Revision: D26883673
fbshipit-source-id: c5ebaf2a327d56cbbe38280c6376c833bcf68b8c
Summary: Implemented `remove`, which, for a typical data source should not be needed. But that would be famous last words and wanted to prevent painting ourselves in a corner, so implemented it. Also because part of the logic is need for the `shift` operation (see next diff), which is much more important.
Reviewed By: priteshrnandgaonkar
Differential Revision: D26883672
fbshipit-source-id: 0dbfcdd3d5a16c4a2d53b0272000d183c67d0034
Summary: ..and some earlier reviews comments has been processed + some fine tuning on the ui
Reviewed By: priteshrnandgaonkar
Differential Revision: D26816559
fbshipit-source-id: adf2586763be185ee8e7cc22b2827ecefe4e4cab
Summary:
This diff implements the remaining features in the logs plugin:
- deeplinking
- merging duplicate rows
The logs plugin source code has now been reduced from originally `935` to `285` LoC. All optimisation code has been removed from the plugin:
* debouncing data processing
* pre-rendering (and storing!) all rows
Finally applied some further styling tweaks and applied some renames to DataTable / DataSource + types finetuning. Some more will follow.
Fixed a emotion warning in unit tests which was pretty annoying.
Reviewed By: passy
Differential Revision: D26666190
fbshipit-source-id: e45e289b4422ebeb46cad927cfc0cfcc9566834f
Summary:
Added styling / coloring to the new logs plugin, to bring it closer to feature completeness. Made the colum headers slightly more compact
Also made the API more foolproof by introducing the `useAssertStableRef` hook, that will protect against accidentally passing in props that would invalidate rendering every time.
Reviewed By: passy
Differential Revision: D26635063
fbshipit-source-id: 60b2af8db3cc3c12d8d25d922cf1735aed91dd2c
Summary:
Per title. Give the user some feedback on why he doesn't see any records (filtering to aggressive vs there are simply no records).
Fixes the old Flipper behavior where tables are merely empty, which always looks kinda broken to me. (Didn't it work, or is there nothing?)
Reviewed By: nikoant
Differential Revision: D26611173
fbshipit-source-id: 7ac798bd7d5c31f6d9fbacf30c6727d2e0e94570
Summary:
This diff has some jak-shaving UX improvements after playing with the DataTable a bit more:
Selection
* deselecting a row from a larger set will make the last selected item the default selection
* re-selecting an item in a single selection will unselect it
Column Filtering
* Introduced button to toggle between filtering on all, nothing, and the values present in the current selection
Column sorting
* The up / down arrows are now inidividually clickable, rather than action as a general toggle
* Title still works as a general toggle between asc / desc / not sorted
Context menu
* I found the context menu for column selection and on the selected rows itself a bit finicky to find and click and not super intuitive for noob users. Merged both menus instead into a single hamburger menu adjacent to the search bar
Reviewed By: passy
Differential Revision: D26580038
fbshipit-source-id: 220f501a1d996acbd51088c08ea866caed768572
Summary:
Make sure DataTable supports multiselection, which works largely the same as before, with a few changes
* shift + click and ctrl + click work as expected
* shift + keyboard navigation works as expected
* drag selection works as expected
* drag selection when dragging accross screens, or Shift icmw with HOME / END / PageUp / PageDown works as expect
* text selection stil works as expected
The context menu items have been updated as well
* filter will filter on all the distinct values in the selection
* copying cells will copy all cells of the given column in the selection, separated by newline
* copying rows / creating a past will create a json array of the selection
Not done yet
- Shifting the selection after inserting rows hasn't been implemented yet
- I'm not entirely happy with the context menu trigger, maybe a hamburger button in the toolbar will be better
Reviewed By: nikoant
Differential Revision: D26548228
fbshipit-source-id: 5d1cddd6aad02ce9649d7980ab3a223e222da893
Summary:
Introduced a context menu for DataTable with some default options. Opted to put it under a visible hovered dropdown instead of on right-click, since this better alings with Ant's design guides (we don't have context clicks anywhere else I think), but if it isn't convincing we can still change it.
Included some default actions, to set up quick filters, and to copy values. For copying rows, implemented it to by default take the JSON from a row, rather than space separated values like in ManagedTable, as many existing plugins customize the onCopy handler to just do that, so it seemed like a better default since it is a richer format. If there are good use cases for the previous behavior, we'll probably find out after the old release :)
Introduced utility to copy text to clipboard in FlipperLib, but decoupled it from Electron.
Didn't include multi select yet, that will be done in a next diff.
Reviewed By: nikoant
Differential Revision: D26513161
fbshipit-source-id: b2b1b20b0a6f4ada9de2566bf6b02171f722c4aa
Summary:
Beyond a search across all columns, it is now possible to specific columns for specific values:
* for a row to be visible, all active column filters need to be matched (e.g. both a filter on time and app has to be satisfied)
* if multiple values within a column are filtered for, these are -or-ed.
* if no value at all within a column is checked, even when they are defined, the column won't take part in filtering
* if there is a general search and column filters, a row has to satisfy both
Filters can be preconfigured, pre-configured filters cannot be removed.
Reseting will reset the filters back to their original
Move `useMemoize` to flipper-plugin
Merged the `ui/utils` and `utils` folder inside `flipper-plugin`
Reviewed By: nikoant
Differential Revision: D26450260
fbshipit-source-id: 11693d5d140cea03cad91c1e0f3438d7b129cf29
Summary: Show a hint during scrolling of the relative offset the user is looking at. This is based on the current virtualisation window, so not 100% accurate, but probably still provides the right signal to the user. See the bottom right of the recording
Reviewed By: nikoant
Differential Revision: D26450261
fbshipit-source-id: 206a860024e346c6b872edc3fc7919019046a6d7
Summary: Introduced search bar and support for custom buttons therein.
Reviewed By: nikoant
Differential Revision: D26338666
fbshipit-source-id: e53cd3c4381e11f5f90c05c92e39a6c8ac2eca65
Summary:
Add support for resizable columns, column sorting, and hiding / showing columns
Moved some utilities from Flipper to flipper-plugin, such as Interactive and LowPassFilter
Split DataTable into two components; DataSourceRenderer which takes care of purely rendering the virtualization, and DataTable that has the Chrome around that, such as column headers, search bar, etc.
Reviewed By: nikoant
Differential Revision: D26321105
fbshipit-source-id: 32b8fc03b4fb97b3af52b23e273c3e5b8cbc4498
Summary:
First rudementary setup of DataTable component that follows a data source. Initially used react-virtuose library, but it performed really badly by doing expensive layout shifts and having troublesome scroll handling. Switched to react-virtual library, which is a bit more level, but much more efficient, and the source code is actually understandable :)
Features:
- hook up to window events of datasource
- high and low prio rendering, based on where the change is happening (should be optimized further)
- sticky scrolling support
- initial column configuration (custom rendering, styling, columns etc will follow in next diffs)
Reviewed By: nikoant
Differential Revision: D26175665
fbshipit-source-id: 224be13b1b32d35e7e01c1dc4198811e2af31102
Summary: This diff introduces the possibility to subscribe to the `output` set of the datasource. It emits three possible event: `reset`, `update`, `shift`.
Reviewed By: jknoxville
Differential Revision: D26100104
fbshipit-source-id: b5fac2289206fab9fb8a437b96ab84034a8b5832
Summary:
Context: https://fb.workplace.com/notes/470523670998369
Per title, also restructured internal logic so that only on intermediate data structure is needed: `output`. This is because we don't need store filter information because we can run that plainly, on the incoming events without storing anything. Sorting is performed directly on `output`. Reverse isn't performed at all, but rather applied lazily when requesting a specific item (so it isn't reflected in output)
Reviewed By: nikoant
Differential Revision: D25976469
fbshipit-source-id: 777f8fdeba09729e19c97c176525b702066b6c2e
Summary:
clear will drop all current records but keep any view preferences. This typically relates to the "clear" button in Flipper.
reset on the other hand will keep the current records, but just reset the view preferences to the default, dropping any filters and sorting criteria
Reviewed By: nikoant
Differential Revision: D25975612
fbshipit-source-id: 5b419f259bffc049daf125090c6754aa6528919b
Summary:
For context see https://fb.workplace.com/notes/470523670998369
This diff adds support for reversing the data collection (in a table this would be used to toggle between ascending and descending sorting). The actual implementation is cleaned up in next diffs and the intermediate collection introduced here is dropped, so this diff is basically only about the unit tests, the implementation is not interesting at this point.
Reviewed By: nikoant
Differential Revision: D25975353
fbshipit-source-id: 2da6da2ed940c2e49e1986696d9b93a7b984db9b
Summary:
For context see https://fb.workplace.com/notes/470523670998369
This diff adds the capability to apply a sorting, and inserts item in a sorted way using binary search in a temporarily intermediate collection. (That collection is optimized away in later diffs, so it is mostly the idea and the tests that are interesting)
Reviewed By: nikoant
Differential Revision: D25953336
fbshipit-source-id: a51b05e25242f0835280ada99798676311511ef0
Summary:
For context see https://fb.workplace.com/notes/470523670998369
This diff introduces the DataSource abstraction, that can store records. If a key is set a key -> record mapping is stored, to make it easy to update existing records using `upsert`, without knowing their exact index.
Internal storage will be slightly altered in upcoming diffs, so don't pay to much attention to that part.
Reviewed By: nikoant
Differential Revision: D25953337
fbshipit-source-id: 1c3b53a2fcf61abaf061946be4af21d2aecc6c6d
Summary:
Bumps [types/jest](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jest) from 26.0.15 to 26.0.20.
<details>
<summary>Commits</summary>
<ul>
<li>See full diff in <a href="https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jest">compare view</a></li>
</ul>
</details>
<br />
[](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `dependabot rebase` will rebase this PR
- `dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `dependabot merge` will merge this PR after your CI passes on it
- `dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `dependabot cancel merge` will cancel a previously requested merge and block automerging
- `dependabot reopen` will reopen this PR if it is closed
- `dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
</details>
Pull Request resolved: https://github.com/facebook/flipper/pull/2024
Reviewed By: mweststrate
Differential Revision: D26843905
Pulled By: passy
fbshipit-source-id: b9537ff8c371ee9660943361e3eeafba23707928
Summary:
Noticed that subsequent `adb logcat` invocations are stateful, and we didn't clear it's state when clearing the logs in memory.
This is a bandaid to get back to the previous behavior. I am wondering whether it wouldn't be just much more intuitive to always clear the logs (`adb logcat -c`) before we start the streaming log listener, so that users are never shown old logs from before flipper was connected?
Also fixed a leak in cleaning up the log listeners that was revealed by the possibility to disable the log plugin
Reviewed By: nikoant
Differential Revision: D26450262
fbshipit-source-id: 5b426e2d0e1fafdbc512d48d22b7bd4f30b61309
Summary:
Added option to bootstrap device plugin in "flipper-pkg".
Changelog: "flipper-pkg init" can now be used to bootstrap device plugins
Reviewed By: mweststrate
Differential Revision: D26389429
fbshipit-source-id: 90773011bd50289004cd747111e1787402840922
Summary:
*Stack summary*: this stack adds ability to manage device plugins in the same way as client plugins: install, update, uninstall, enable (star) and disable (unstar) them.
*Diff summary*: changed the way how plugin compatibility with devices is checked from dynamic call to "supportsDevice" to static checks of "supportedDevices" metadata property which make it possible to check compatibility without even downloading plugin from Marketplace.
Changelog: plugin compatibility with devices is now checked according to metadata in property "supportedDevices" in package.json
Reviewed By: mweststrate
Differential Revision: D26315848
fbshipit-source-id: 6e4b052c4ea0507ee185fc17999b6211cdb11093
Summary:
*Stack summary*: this stack refactors plugin management actions to perform them in a dispatcher rather than in the root reducer (store.tsx) as all of these actions has side effects. To do that, we store requested plugin management actions (install/update/uninstall, star/unstar) in a queue which is then handled by pluginManager dispatcher. This dispatcher then dispatches all required state updates.
*Diff summary*: refactored "uninstall plugin" operation to perform it in pluginManager dispatcher
Reviewed By: mweststrate
Differential Revision: D26166198
fbshipit-source-id: d74a1d690102d9036c6d3d8612d2428f5ecef4e6
Summary:
Changelog: Fixed issue where a Flipper crash would result in an entirely blank screen, rather than a useful error message.
While debugging another issue, discovered that React errors that happen outside a Plugin aren't caught at all, resulting in the infamous gray screen of deaths. This was the case because no error boundary has been set up for our Chrome, and since React 16 the default error handling has becoming rendering blank, rather than freezing. See https://reactjs.org/docs/error-boundaries.html#new-behavior-for-uncaught-errors.
Thanks to ant.design styling this decently was trivial :). But sadly involved a component class since error boundaries are not yet available as hook.
With these changes the errors should also end up more readably in our monitoring.
Reviewed By: nikoant
Differential Revision: D26422666
fbshipit-source-id: 6c0f8611c80a4a5e0d7e61d58afcf5eabe410e57
Summary:
Minor code cleanup to avoid future confusion:
- archived: a device that was imported from a Flipper trace, and only has persisted state
- (dis)connected: a real stateful device that might or might not have an active connection
Reviewed By: nikoant
Differential Revision: D26275459
fbshipit-source-id: eba554b37c39711e367c3795ff4456329a303c22
Summary:
This diff addresses two problems:
1. Since clients plugins can be active beyond having a connection, we have to make it possible for plugin authors to check if they are connected before they make a call.
2. if there is a custom `exportPersistedState`, plugins should be able to skip making calls if the device has disconnected.
Introducing this change makes it possible to interact with a reasonable level with disconnected clients, and makes it possible to create Flipper traces for disconnected clients.
Note that both items were already problems before supporting offline clients; as there can be a noticeable delay between disconnecting and Flipper detecting that (i've seen up to 30 secs). What happend previously in those cases is that the export would simply hang, as would other user interactions, as loosing the connection in the middle of a process would cause the promise chains to be neither rejected or resolved, which is pretty iffy.
Before this diff, trying to export a disconnected device would hang forever like:
{F369600601}
Reviewed By: nikoant
Differential Revision: D26250895
fbshipit-source-id: 177624a116883c3cba14390cd0fe164e243bb97c
Summary:
It should be possible to exported disconnected devices, so that flipper traces / support form reports can be created from them. This diff introduces this functionality. Support for plugins with custom export logic is introduced in a later diff.
Issues fixed in this diff:
- don't try to take a screenshot for a disconnected device (this would hang forever)
- device plugins were always exported, regardless whether the user did select them or not
- sandy plugins were never part of exported disconnected clients
- increased the amount of data exported for device logs to ~10 MB. This makes more sense now as the logs will no longer be included in all cases
- fixed issue where are plugins would appear to be enabled after the client disconnected (this bug is the result of some unfortunate naming of `isArchived` vs `isConnected` semantics. Will clean up those names in a later diff.
Changelog: It is now possible to create a Flipper trace for disconnected devices and apps
Reviewed By: nikoant
Differential Revision: D26250894
fbshipit-source-id: 4dd0ec0cb152b1a8f649c31913e80efc25bcc5dd
Summary:
Introduced `isConnected` flag on device and plugin client to reflect whether a connection is still available for the plugins, or that they have been disconnected.
Potentially we could expose the (readonly) `connected` state atom for this as well, or an `onDisconnect` event for device pugins, to create a responsive UI, but there might be no need for that, in which case this suffices.
Reviewed By: nikoant
Differential Revision: D26249346
fbshipit-source-id: b8486713fdf2fcd520488ce54f771bd038fd13f8
Summary:
This diff stack introduces support for keeping devices and clients around after they have disconnected. This is a pretty important debugging improvement, that will allow inspecting a device / app after it crashed for example.
This feature existed partially before, but only supported Android, and only support plugins with persisted state; as it replace the current device with an archived version of the same device. In practice this didn't work really well, as most plugins would not be available, and all non-persisted state would be lost.
This diff makes sure we can keep devices around after disconnecting, the next one will keep the clients around as well. And explain some code choices in more detail.
Note that `Device.isArchived` was an overloaded term before, and even more now (both representing imported and disconnected devices), will address that in a later diff.
https://github.com/facebook/flipper/issues/1460https://github.com/facebook/flipper/issues/812https://github.com/facebook/flipper/issues/1487
Changelog: iOS and Android devices will preserve their state after being disconnected
Reviewed By: nikoant
Differential Revision: D26224310
fbshipit-source-id: 7dfc93c2a109a51c2880ec212a00463bc8d32041
Summary:
Logs were stored hardcoded on the Device object first, this diff makes it normal plugin state.
This makes sure that we can use the same abstractions as in all plugins that store large data sets, and that we can leverage the upcoming DataSource abstraction.
Reviewed By: nikoant
Differential Revision: D26127243
fbshipit-source-id: 7c386a615fa7989f35ba0df5b7c1d218d37b57a2
Summary: Per title, this allows for pre-processing data after it is deserialized and before it is stored in the plugin
Reviewed By: nikoant
Differential Revision: D26126423
fbshipit-source-id: bc08a6ab205d2a0d551515563cd85a197595ddb2
Summary:
Sandy plugins can now set up an `onExport` handler to enable customizing the export format of a plugin: `client.onExport(callback: (idler, onStatusMessage) => Promise<state>)`
Import will be done in next diff
Reviewed By: nikoant
Differential Revision: D26124440
fbshipit-source-id: c787c79d929aa8fb484f15a9340d7c87545793cb
Summary: Plugin metadata format extended to include type of each plugin (client / device) and list of supported devices (android/ios/..., emulator/physical, etc). This will allow to detect plugins supported by device even if they are not installed and only available on Marketplace.
Reviewed By: mweststrate
Differential Revision: D26073531
fbshipit-source-id: e331f1be1af1046cd4220a286a1d52378c26cc53
Summary:
Immer had security issues. Noobs.
Only the patches feature was affected, which at this moment isn't used by any known plugin, but just in case.
Reviewed By: passy
Differential Revision: D25993961
fbshipit-source-id: 9bfe7065c90865d912ebcfd14c1117697bf714f7
Summary:
I've re-designed interfaces describing plugins as I found that mental overhead working with them became too expensive because of slightly flawed design. However this cascaded changes in many files so you can see how extensively these interfaces used in our codebase.
Before this change we had one interface PluginDetails which described three different entities: 1) plugins installed on the disk 2) plugins bundled into Flipper 3) plugins available on Marketplace. It's hard to use this "general" PluginDetails interface because of this as you always need to think about all three use cases everywhere.
After this change we have 3 separate interfaces: InstalledPluginDetails, BundledPluginDetails and DownloadablePluginDetails and things became much type-safer now.
Reviewed By: mweststrate
Differential Revision: D25530383
fbshipit-source-id: b93593916a980c04e36dc6ffa168797645a0ff9c
Summary:
Pull Request resolved: https://github.com/facebook/flipper/pull/1761
Plugin installed logic is quite complex and heavily relies on file system. So I added integration tests which uses in-memory file system. Here I've covered complex functions added in the previous diff of this stack.
Reviewed By: mweststrate
Differential Revision: D25393473
fbshipit-source-id: a823eb7d6707152a1c3717a6da25b6beee85801d
Summary:
Introduced API to replace the deprecated `selectPlugin` in Sandy.
The API can be used to navigate from `device plugin -> device plugin`, or` client plugin -> device / client plugin`
Introduced `isPluginAvailable` as well, so that the user interaction an be fine tuned in case the plugin is not disabled.
Reviewed By: jknoxville
Differential Revision: D25422370
fbshipit-source-id: c6c603f1c68e6291280b3d0883e474448754ded1
Summary:
`activatePlugin` events where not recorded correctly for Sandy plugins. Although the starting measuerments is fired the `connections` reducer, the completing event fires from `PluginContainer`. Since this is done as part of a ref-update (!!), see [here](https://www.internalfb.com/intern/diffusion/FBS/browse/master/xplat/sonar/desktop/app/src/PluginContainer.tsx?commit=65a625ea9941&lines=155), and the ref to the corresponding element is not set for Sandy plugins, the event was never marked as completed.
Fixed this by making it part of the `activate` life-cycle event of Sandy plugins.
Reviewed By: passy
Differential Revision: D25421537
fbshipit-source-id: 5cbfeb91cc12e4520fa271bab24034094d7ddb39
Summary: When exposing new top-level APIs from `flipper-plugin`, they should be documented. Added a unit test to enforce this and added documentation for all missing APIs.
Reviewed By: passy
Differential Revision: D25421401
fbshipit-source-id: f5cafc1881de846c8a5dd86e5d094ebd27a66f2a
Summary: In the Flipper chrome there is a lot of `logger={logger}` prop drilling. Let's not do that anymore in the future by using a proper hook, which is exposed from `flipper-plugin`.
Reviewed By: passy
Differential Revision: D25421304
fbshipit-source-id: 01ec8563c67f7e2fac359c2f8216eba722bff8d9
Summary: Make sure that GKs can be used in pure sandy plugins.
Reviewed By: jknoxville
Differential Revision: D25368358
fbshipit-source-id: c7c6aa4ecf0443cb3b5d90e22e8aca9a73a69389
