I had 1024×1024 tensors that I pickled and they weighted 900 Mb which I found very odd.
Here is the very big tensor for reproducibility purposes (https://we.tl/t-ERWsb81HFv – I realize downloading files from a stranger seems dangerous, sorry, I don’t really know how I can share the file otherwise).
When running this:
import pickle with open("test3.pkl", "rb") as f: image_2d = pickle.load(f) image_2d.dtype, image_2d.shape
I get
(torch.float32, torch.Size([1024, 1024]))
which seems normal.
If I dump the file, it stays at 900 Mb.
If I do
image_2d = image_2d / 1
I still have a
(torch.float32, torch.Size([1024, 1024]))
but when I dump the file it goes to 4 Mb.
What am I doing wrong?
submitted by /u/Lindayz
[link] [comments]
r/learnpython I had 1024×1024 tensors that I pickled and they weighted 900 Mb which I found very odd. Here is the very big tensor for reproducibility purposes (https://we.tl/t-ERWsb81HFv – I realize downloading files from a stranger seems dangerous, sorry, I don’t really know how I can share the file otherwise). When running this: import pickle with open(“test3.pkl”, “rb”) as f: image_2d = pickle.load(f) image_2d.dtype, image_2d.shape I get (torch.float32, torch.Size([1024, 1024])) which seems normal. If I dump the file, it stays at 900 Mb. If I do image_2d = image_2d / 1 I still have a (torch.float32, torch.Size([1024, 1024])) but when I dump the file it goes to 4 Mb. What am I doing wrong? submitted by /u/Lindayz [link] [comments]
I had 1024×1024 tensors that I pickled and they weighted 900 Mb which I found very odd.
Here is the very big tensor for reproducibility purposes (https://we.tl/t-ERWsb81HFv – I realize downloading files from a stranger seems dangerous, sorry, I don’t really know how I can share the file otherwise).
When running this:
import pickle with open("test3.pkl", "rb") as f: image_2d = pickle.load(f) image_2d.dtype, image_2d.shape
I get
(torch.float32, torch.Size([1024, 1024]))
which seems normal.
If I dump the file, it stays at 900 Mb.
If I do
image_2d = image_2d / 1
I still have a
(torch.float32, torch.Size([1024, 1024]))
but when I dump the file it goes to 4 Mb.
What am I doing wrong?
submitted by /u/Lindayz
[link] [comments]