Summary:
Changelog: Fixed an issue where Flipper would crash when decoding large partial requests.
The current processing of partial requests assumes that the proviced base64 string is always an utf-8 string, which is incorrect as it might contain binary data as well. This causes `atob` build in to throw errors when trying to decode binary base64 strings with the following exception:
{F538782963}
However, what is worse, if those strings were larger than ~2 mb, it would completely crash Electron rather than on the JS level, with reports like:
```
Crashed Thread: 0 CrRendererMain Dispatch queue: com.apple.main-thread
Exception Type: EXC_BAD_ACCESS (SIGSEGV)
Exception Codes: EXC_I386_GPFLT
Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Segmentation fault: 11
Termination Reason: Namespace SIGNAL, Code 0xb
Terminating Process: exc handler [85268]
Thread 0 Crashed:: CrRendererMain Dispatch queue: com.apple.main-thread
0 com.github.Electron.framework 0x000000011155b16f v8::internal::SetupIsolateDelegate::SetupHeap(v8::internal::Heap*) + 22324575
1 com.github.Electron.framework 0x000000011155e811 v8::internal::SetupIsolateDelegate::SetupHeap(v8::internal::Heap*) + 22338561
2 com.github.Electron.framework 0x00000001117e2e62 v8::internal::SetupIsolateDelegate::SetupHeap(v8::internal::Heap*) + 24978002
3 com.github.Electron.framework 0x000000010fa32660 v8::internal::ClassScope::ResolvePrivateNamesPartially() + 14944
4 com.github.Electron.framework 0x000000010fa322b5 v8::internal::ClassScope::ResolvePrivateNamesPartially() + 14005
5 com.github.Electron.framework 0x000000010fa31933 v8::internal::ClassScope::ResolvePrivateNamesPartially() + 11571
6 com.github.Electron.framework 0x000000011007ef58 v8::internal::SetupIsolateDelegate::SetupHeap(v8::internal::Heap*) + 451400
```
Reproduced this JS issue by lowering the `MAX_BODY_SIZE_IN_BYTES` in `NetworkFlipperPlugin.java` to 10KB, which causes all requests to be processed as partials.
Reproducing the the Electron crash is a lot harder, as it requires a surface that makes large, binary requests (more than a few mb), that is still intercepted by the Network layer. The best example I could find is sending large pictures or videos through a messenger for android chat. In that case it is still hard to produce due to caching though.
Fun fact, you can crash your own flipper and get the above crash by running this command:
`btoa(require("fs").readFileSync("/Users/mweststrate/Desktop/Screen Recording 2021-03-24 at 16.08.27 crop.mov", "binary"))`, where the provided file must be a few mb's large (this one is 10).
A result of fixing this issue, is that images that were send as partials can now be correctly previewed in the Network plugin again.
Reviewed By: jknoxville
Differential Revision: D27302961
fbshipit-source-id: 1ac86840f7268062bb59c789f3904537df3c51fa
Summary: After calling "bundle-all-plugins" locally, "yarn test" is failing with obscure message, because some tests are trying to import built bundles instead of "index.tsx". This diff fixes that.
Reviewed By: passy
Differential Revision: D26986246
fbshipit-source-id: cffe988dc642e2c5d2b2028581cd162350186e0c
Summary:
Changelog: [Network] Non-binary request are not properly utf-8 decoded on both iOS and Android, both when gzipped and when not gzipped
This diff fixes a long standing / ping-pong issue regarding network decoding differences between
* iOS vs Android
* binary vs utf-8
* gzipped vs uncompressed
The changes aren't too big, but the underlying investigating is :)
The primary contribution to this diff is:
First, adding test cases for know problematic cases. This is done by grabbing the messages that are send from the flipper client to flipper using the flipper messages plugin. This is the base64 data that is stored in the `.txt` files. Beyond that, for all tests public endpoints are used, so that we can both get a hold of the raw original files, and how we expect them to be displayed in flipper.
For testing a simple RN app was build, with a button that fires a bunch requests. The first 3 are captured in unit tests, the last one is not idempotent, but a case reported in #1466, so just left it there as manual verification.
```
const fetchData = async () => {
await fetch(
'https://raw.githubusercontent.com/SangKa/MobX-Docs-CN/master/docs/donating.md',
{
headers: {
'Accept-Encoding': 'identity', // signals that we don't want gzip
},
},
);
await fetch('https://reactnative.dev/img/tiny_logo.png?x=' + Math.random());
await fetch(
'https://raw.githubusercontent.com/SangKa/MobX-Docs-CN/master/docs/donating.md',
);
await fetch(
'https://ex.ke.com/sdk/recommend/html/100001314?hdicCityId=110000¶mMap[source]=&id=100001314&mediumId=100000037&elementId=&resblockId=1111027381003&templateConfig=%5Bobject%20Object%5D&fbExpoId=346620976471638017&fbQueryId=&required400=true&unique=1111027381003&parentSceneId=',
);
};
```
The second contribution of this diff is that it doesn't use weird URLencoder hacks to convert base64 to utf8, but rather a proper library. The problem with our original solution, using `atob` is that it converts to ASCII, not to utf-8, which is the source of the original bugs. See for more background on this: https://www.npmjs.com/package/js-base64#decode-vs-atob-and-encode-vs-btoa-
Solves:
https://github.com/facebook/flipper/issues/1466https://github.com/facebook/flipper/pull/1541https://github.com/facebook/flipper/issues/1458
Supersedes D23837750
Future work: we don't inspect the `content-type=xxx;charset` header yet, which we should do for less common encodings, to make sure that they get displayed correctly as well
Future work: in feature like copy data and curl, we always call decode body, without check if we are actually dealing with non-binary data. Probably it is better to keep binary data in base64, rather than decoding it, as that will assume the data is an utf-8 string, which might fail.
An assumption in these changes is that binary data is never gzipped, which is generally correct; gzip is not applied by webserver to things like images, as it would increase, not decrease their size, and waste a lot of computation power.
Reviewed By: cekkaewnumchai
Differential Revision: D23403095
fbshipit-source-id: 5099cc4a7503f0f63bd10585dc6590ba893f3dde
Summary:
It's common for responses to be completely missing in the network inspector. This is because they are larger than can be serialized in one go on some devices, so we drop all messages larger than 1MB.
This changes the android client to send large responses in individually serialized batches. This way we avoid running out of memory and can still send arbitrarily large payloads.
Changelog: Android network inspector can now handle responses large than 1MB.
Reviewed By: passy
Differential Revision: D22999905
fbshipit-source-id: ff4eb8fa72a7e42ea90d12ffe0f20c6d1e58b7e5
