Minor doc improvements

Summary: per title

Reviewed By: muraziz

Differential Revision: D28538087

fbshipit-source-id: 322558d909fb89cd297b56d6aaf3de98e20b4ec1
This commit is contained in:
Michel Weststrate
2021-05-19 08:15:57 -07:00
committed by Facebook GitHub Bot
parent f0dc199d38
commit c897d92df8
3 changed files with 106 additions and 12 deletions

View File

@@ -1,6 +1,25 @@
# DataSource
_Library to power streamig data visualisations_
_Library to power streamig data visualisations as used in Facebook's Flipper_
## Installation
`yarn add flipper-data-source`
## Links
* [DataSource API documentation](https://fbflipper.com/docs/extending/flipper-plugin#createdatasource)
* [Demo sandbox](https://codesandbox.io/s/flipper-datasource-demo-iy0tq)
* [Demo setup recording](https://www.youtube.com/watch?v=stL66GByQU0)
* Introduction talk (TODO)
* [Source code](https://github.com/facebook/flipper/tree/master/desktop/flipper-plugin/src/data-source)
* [Lightning talk using DataSource in Logs view](https://fb.workplace.com/groups/427492358561913/permalink/432720091372473/) [FB-Only]
* [DataSource project plan](https://fb.quip.com/noJDArpLF7Fe) [FB-Only]
* [Demo project](https://www.internalfb.com/code/flipper/src/fbsource/xplat/sonar/facebook/data-source-demo/) [FB-Only]
![DataSource powers the ADB logs in Flipper](./img/flipper-logs.png)
_flipper-data-source powering the ADB logs view in Flipper_
## Contstraints & Benefits
@@ -20,13 +39,6 @@ This library is designed with the following constraints in mind:
After applying this abstraction (right two sections), in Flipper we saw a 10-20 fold framerate increase while displaying 100K log items, to which new records where added at a rate of ~50/sec. The above image is taken while tailing the logs (so it continuesly scrolls with the arrival of new data), while a search filter is also active. In the first two sections, scripting ate away most of the CPU, result in just a couple of frames per second. After applying these changes CPU time is primarily spend on cranking out more frames resulting in a smooth rather than stuttering scroll (see the lightning talk below for a demo). On top of that the base situation uses a fixed row-height, while the new situation supports text wrapping.
## Links
* [DataSource API documentation](https://fbflipper.com/docs/extending/flipper-plugin#createdatasource)
* [DataSource project plan](https://fb.quip.com/noJDArpLF7Fe)
* Introduction talk (TODO)
* [Lightning talk using DataSource in Logs view](https://fb.workplace.com/groups/427492358561913/permalink/432720091372473/)
## More detailed explanation
![FSRW pipeline](img/FSRW.png)
@@ -64,13 +76,94 @@ Typically this component is used as underlying abstraction for a Table represent
A simplified (and not very efficient) render for DataSource that doens't use virtualization. Use this as basic for a natural growing representaiton.
### DataSourceChartRenderer
## Example usage
### Using DataSource in a table
Excerpt from https://codesandbox.io/s/flipper-datasource-demo-iy0tq?file=/src/CBDataSource.tsx:
```typescript
export type RowData = {
product_id: string;
price: number;
};
export function CBDataSource() {
// create a DataSource that will hold our data
const [dataSource] = useState(() => new DataSource<RowData>(undefined));
// search / sort / tail preferences of the user
const [search, setSearch] = useState("");
const [sorted, setSorted] = useState(false);
const [sticky, setSticky] = useState(false);
// listen to coin stream
useEffect(() => {
return streamCoinbase((event) => {
dataSource.append({
product_id: event.product_id,
price: parseFloat(event.price)
});
});
}, []);
// apply filter
useEffect(() => {
dataSource.view.setFilter(
search ? (r) => r.product_id.includes(search) : undefined
);
}, [search]);
// apply sort (by field or function)
useEffect(() => {
dataSource.view.setSortBy(sorted ? "price" : undefined);
}, [sorted]);
// rendering
return (
<div>
{/* toolbar omitted */}
<div className="table">
<DataSourceRendererVirtual
dataSource={dataSource}
itemRenderer={rowRenderer}
autoScroll={sticky}
/>
</div>
</div>
);
}
function rowRenderer(row: RowData) {
return <Row row={row} />;
}
function Row({ row }: { row: RowData }) {
return (
<div className="row">
<div>{row.product_id}</div>
<div>{row.price}</div>
</div>
);
}
```
### Using DataSource in a Chart
Experimental. See: https://codesandbox.io/s/flipper-datasource-demo-iy0tq?file=/src/DataSourceChart.tsx
## Future work
Project setup:
* [ ] **Give this thing a proper name**
* [ ] **Leverage React concurrent mode**: Currently there is custom scheduler logic to handle high- and low- (outside window) priority updates. In principle this could probably be achieved through React concurrent mode as well, but ANT.design (which is used in Flipper) doesn't support it yet.
* [ ] **Move to top-level Flipper or stand alon package**
* [ ] **Reduce build size**. Currently half lodash is baked in, but basically we only need it's binary sort function :).
Features:
* [ ] **Support multiple DataSourceView's per DataSource**: Currently there is a one view per source limitation because we didn't need more yet.
* [ ] **Break up operations that process the full data set in smaller tasks**: There are several operations that process the full data set, for example changing the sort / filter criteria. Currently this is done synchronously (and we debounce changing the filter), in the future we will split up the filtering in smaller taks to make it efficient. But we don't have a way to efficiently break down sorting into smaller tasks as using insertion sorting is 20x slower than the native sorting mechanism if the full data set needs to be processed.
* [ ] **Publish as open source / standalone package**
* [ ] **Reduce build size**. Currently half lodash is baked in, but basically we only need it's binary sort function :).
* [ ] **Add built-in support for downsampling data**
* [ ] **Leverage React concurrent mode**: Currently there is custom scheduler logic to handle high- and low- (outside window) priority updates. In principle this could probably be achieved through React concurrent mode as well, but ANT.design (which is used in Flipper) doesn't support it yet.

Binary file not shown.

After

Width:  |  Height:  |  Size: 165 KiB

View File

@@ -3,6 +3,7 @@
"version": "0.0.1",
"description": "Library to power streamig data visualisations",
"repository": "https://github.com/facebook/flipper",
"homepage": "https://github.com/facebook/flipper/blob/master/desktop/flipper-plugin/src/data-source/README.md",
"author": "Facebook Inc",
"license": "MIT",
"source": "index.tsx",