Minor doc improvements
Summary: per title Reviewed By: muraziz Differential Revision: D28538087 fbshipit-source-id: 322558d909fb89cd297b56d6aaf3de98e20b4ec1
This commit is contained in:
committed by
Facebook GitHub Bot
parent
f0dc199d38
commit
c897d92df8
@@ -1,6 +1,25 @@
|
|||||||
# DataSource
|
# DataSource
|
||||||
|
|
||||||
_Library to power streamig data visualisations_
|
_Library to power streamig data visualisations as used in Facebook's Flipper_
|
||||||
|
|
||||||
|
## Installation
|
||||||
|
|
||||||
|
`yarn add flipper-data-source`
|
||||||
|
|
||||||
|
## Links
|
||||||
|
|
||||||
|
* [DataSource API documentation](https://fbflipper.com/docs/extending/flipper-plugin#createdatasource)
|
||||||
|
* [Demo sandbox](https://codesandbox.io/s/flipper-datasource-demo-iy0tq)
|
||||||
|
* [Demo setup recording](https://www.youtube.com/watch?v=stL66GByQU0)
|
||||||
|
* Introduction talk (TODO)
|
||||||
|
* [Source code](https://github.com/facebook/flipper/tree/master/desktop/flipper-plugin/src/data-source)
|
||||||
|
* [Lightning talk using DataSource in Logs view](https://fb.workplace.com/groups/427492358561913/permalink/432720091372473/) [FB-Only]
|
||||||
|
* [DataSource project plan](https://fb.quip.com/noJDArpLF7Fe) [FB-Only]
|
||||||
|
* [Demo project](https://www.internalfb.com/code/flipper/src/fbsource/xplat/sonar/facebook/data-source-demo/) [FB-Only]
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
|
_flipper-data-source powering the ADB logs view in Flipper_
|
||||||
|
|
||||||
## Contstraints & Benefits
|
## Contstraints & Benefits
|
||||||
|
|
||||||
@@ -20,13 +39,6 @@ This library is designed with the following constraints in mind:
|
|||||||
|
|
||||||
After applying this abstraction (right two sections), in Flipper we saw a 10-20 fold framerate increase while displaying 100K log items, to which new records where added at a rate of ~50/sec. The above image is taken while tailing the logs (so it continuesly scrolls with the arrival of new data), while a search filter is also active. In the first two sections, scripting ate away most of the CPU, result in just a couple of frames per second. After applying these changes CPU time is primarily spend on cranking out more frames resulting in a smooth rather than stuttering scroll (see the lightning talk below for a demo). On top of that the base situation uses a fixed row-height, while the new situation supports text wrapping.
|
After applying this abstraction (right two sections), in Flipper we saw a 10-20 fold framerate increase while displaying 100K log items, to which new records where added at a rate of ~50/sec. The above image is taken while tailing the logs (so it continuesly scrolls with the arrival of new data), while a search filter is also active. In the first two sections, scripting ate away most of the CPU, result in just a couple of frames per second. After applying these changes CPU time is primarily spend on cranking out more frames resulting in a smooth rather than stuttering scroll (see the lightning talk below for a demo). On top of that the base situation uses a fixed row-height, while the new situation supports text wrapping.
|
||||||
|
|
||||||
## Links
|
|
||||||
|
|
||||||
* [DataSource API documentation](https://fbflipper.com/docs/extending/flipper-plugin#createdatasource)
|
|
||||||
* [DataSource project plan](https://fb.quip.com/noJDArpLF7Fe)
|
|
||||||
* Introduction talk (TODO)
|
|
||||||
* [Lightning talk using DataSource in Logs view](https://fb.workplace.com/groups/427492358561913/permalink/432720091372473/)
|
|
||||||
|
|
||||||
## More detailed explanation
|
## More detailed explanation
|
||||||
|
|
||||||

|

|
||||||
@@ -64,13 +76,94 @@ Typically this component is used as underlying abstraction for a Table represent
|
|||||||
|
|
||||||
A simplified (and not very efficient) render for DataSource that doens't use virtualization. Use this as basic for a natural growing representaiton.
|
A simplified (and not very efficient) render for DataSource that doens't use virtualization. Use this as basic for a natural growing representaiton.
|
||||||
|
|
||||||
### DataSourceChartRenderer
|
## Example usage
|
||||||
|
|
||||||
|
### Using DataSource in a table
|
||||||
|
|
||||||
|
Excerpt from https://codesandbox.io/s/flipper-datasource-demo-iy0tq?file=/src/CBDataSource.tsx:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
export type RowData = {
|
||||||
|
product_id: string;
|
||||||
|
price: number;
|
||||||
|
};
|
||||||
|
|
||||||
|
export function CBDataSource() {
|
||||||
|
// create a DataSource that will hold our data
|
||||||
|
const [dataSource] = useState(() => new DataSource<RowData>(undefined));
|
||||||
|
|
||||||
|
// search / sort / tail preferences of the user
|
||||||
|
const [search, setSearch] = useState("");
|
||||||
|
const [sorted, setSorted] = useState(false);
|
||||||
|
const [sticky, setSticky] = useState(false);
|
||||||
|
|
||||||
|
// listen to coin stream
|
||||||
|
useEffect(() => {
|
||||||
|
return streamCoinbase((event) => {
|
||||||
|
dataSource.append({
|
||||||
|
product_id: event.product_id,
|
||||||
|
price: parseFloat(event.price)
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
// apply filter
|
||||||
|
useEffect(() => {
|
||||||
|
dataSource.view.setFilter(
|
||||||
|
search ? (r) => r.product_id.includes(search) : undefined
|
||||||
|
);
|
||||||
|
}, [search]);
|
||||||
|
// apply sort (by field or function)
|
||||||
|
useEffect(() => {
|
||||||
|
dataSource.view.setSortBy(sorted ? "price" : undefined);
|
||||||
|
}, [sorted]);
|
||||||
|
|
||||||
|
// rendering
|
||||||
|
return (
|
||||||
|
<div>
|
||||||
|
{/* toolbar omitted */}
|
||||||
|
<div className="table">
|
||||||
|
<DataSourceRendererVirtual
|
||||||
|
dataSource={dataSource}
|
||||||
|
itemRenderer={rowRenderer}
|
||||||
|
autoScroll={sticky}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function rowRenderer(row: RowData) {
|
||||||
|
return <Row row={row} />;
|
||||||
|
}
|
||||||
|
|
||||||
|
function Row({ row }: { row: RowData }) {
|
||||||
|
return (
|
||||||
|
<div className="row">
|
||||||
|
<div>{row.product_id}</div>
|
||||||
|
<div>{row.price}</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
### Using DataSource in a Chart
|
||||||
|
|
||||||
|
Experimental. See: https://codesandbox.io/s/flipper-datasource-demo-iy0tq?file=/src/DataSourceChart.tsx
|
||||||
|
|
||||||
|
|
||||||
## Future work
|
## Future work
|
||||||
|
|
||||||
|
Project setup:
|
||||||
|
|
||||||
* [ ] **Give this thing a proper name**
|
* [ ] **Give this thing a proper name**
|
||||||
* [ ] **Leverage React concurrent mode**: Currently there is custom scheduler logic to handle high- and low- (outside window) priority updates. In principle this could probably be achieved through React concurrent mode as well, but ANT.design (which is used in Flipper) doesn't support it yet.
|
* [ ] **Move to top-level Flipper or stand alon package**
|
||||||
|
* [ ] **Reduce build size**. Currently half lodash is baked in, but basically we only need it's binary sort function :).
|
||||||
|
|
||||||
|
Features:
|
||||||
|
|
||||||
* [ ] **Support multiple DataSourceView's per DataSource**: Currently there is a one view per source limitation because we didn't need more yet.
|
* [ ] **Support multiple DataSourceView's per DataSource**: Currently there is a one view per source limitation because we didn't need more yet.
|
||||||
* [ ] **Break up operations that process the full data set in smaller tasks**: There are several operations that process the full data set, for example changing the sort / filter criteria. Currently this is done synchronously (and we debounce changing the filter), in the future we will split up the filtering in smaller taks to make it efficient. But we don't have a way to efficiently break down sorting into smaller tasks as using insertion sorting is 20x slower than the native sorting mechanism if the full data set needs to be processed.
|
* [ ] **Break up operations that process the full data set in smaller tasks**: There are several operations that process the full data set, for example changing the sort / filter criteria. Currently this is done synchronously (and we debounce changing the filter), in the future we will split up the filtering in smaller taks to make it efficient. But we don't have a way to efficiently break down sorting into smaller tasks as using insertion sorting is 20x slower than the native sorting mechanism if the full data set needs to be processed.
|
||||||
* [ ] **Publish as open source / standalone package**
|
* [ ] **Add built-in support for downsampling data**
|
||||||
* [ ] **Reduce build size**. Currently half lodash is baked in, but basically we only need it's binary sort function :).
|
* [ ] **Leverage React concurrent mode**: Currently there is custom scheduler logic to handle high- and low- (outside window) priority updates. In principle this could probably be achieved through React concurrent mode as well, but ANT.design (which is used in Flipper) doesn't support it yet.
|
||||||
|
|||||||
BIN
desktop/flipper-plugin/src/data-source/img/flipper-logs.png
Normal file
BIN
desktop/flipper-plugin/src/data-source/img/flipper-logs.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 165 KiB |
@@ -3,6 +3,7 @@
|
|||||||
"version": "0.0.1",
|
"version": "0.0.1",
|
||||||
"description": "Library to power streamig data visualisations",
|
"description": "Library to power streamig data visualisations",
|
||||||
"repository": "https://github.com/facebook/flipper",
|
"repository": "https://github.com/facebook/flipper",
|
||||||
|
"homepage": "https://github.com/facebook/flipper/blob/master/desktop/flipper-plugin/src/data-source/README.md",
|
||||||
"author": "Facebook Inc",
|
"author": "Facebook Inc",
|
||||||
"license": "MIT",
|
"license": "MIT",
|
||||||
"source": "index.tsx",
|
"source": "index.tsx",
|
||||||
|
|||||||
Reference in New Issue
Block a user