Pandas: Loading excel data is causing significant memory overhead /u/lofi_thoughts Python Education

Pandas: Loading excel data is causing significant memory overhead /u/lofi_thoughts Python Education

I have an excel file around 70MB and by just even loading it is causing memory to jump to 600mb. I wished if I had a csv file so I could have loaded the file in chunks line by line but that’s not the case here. I will be having am excel file coming that woul be around 2GB and I have to process it, so any tips/tricks to get around this memory issue?

Here is my code below:

def test_excel(excel_file): try: df = pd.read_excel(excel_file, usecols=[0, 5, 7, 12]) print(f"Successfully done") except Exception as e: print(f"Error: {e}") # Example usage: test_excel("final.xlsx") 

submitted by /u/lofi_thoughts
[link] [comments]

​r/learnpython I have an excel file around 70MB and by just even loading it is causing memory to jump to 600mb. I wished if I had a csv file so I could have loaded the file in chunks line by line but that’s not the case here. I will be having am excel file coming that woul be around 2GB and I have to process it, so any tips/tricks to get around this memory issue? Here is my code below: def test_excel(excel_file): try: df = pd.read_excel(excel_file, usecols=[0, 5, 7, 12]) print(f”Successfully done”) except Exception as e: print(f”Error: {e}”) # Example usage: test_excel(“final.xlsx”) submitted by /u/lofi_thoughts [link] [comments] 

I have an excel file around 70MB and by just even loading it is causing memory to jump to 600mb. I wished if I had a csv file so I could have loaded the file in chunks line by line but that’s not the case here. I will be having am excel file coming that woul be around 2GB and I have to process it, so any tips/tricks to get around this memory issue?

Here is my code below:

def test_excel(excel_file): try: df = pd.read_excel(excel_file, usecols=[0, 5, 7, 12]) print(f"Successfully done") except Exception as e: print(f"Error: {e}") # Example usage: test_excel("final.xlsx") 

submitted by /u/lofi_thoughts
[link] [comments]  I have an excel file around 70MB and by just even loading it is causing memory to jump to 600mb. I wished if I had a csv file so I could have loaded the file in chunks line by line but that’s not the case here. I will be having am excel file coming that woul be around 2GB and I have to process it, so any tips/tricks to get around this memory issue? Here is my code below: def test_excel(excel_file): try: df = pd.read_excel(excel_file, usecols=[0, 5, 7, 12]) print(f”Successfully done”) except Exception as e: print(f”Error: {e}”) # Example usage: test_excel(“final.xlsx”) submitted by /u/lofi_thoughts [link] [comments]

Read more

Help with assignment please! /u/Jealous_League2870 Python Education

Help with assignment please! /u/Jealous_League2870 Python Education

Hi everyone,

I would really need someone’s help with an assignment I have! Can’t post it here but can send it via email / anywhere else. This assignment requires both knowledge in Python and excel.

Thanks everyone and happy holidays!!

submitted by /u/Jealous_League2870
[link] [comments]

​r/learnpython Hi everyone, I would really need someone’s help with an assignment I have! Can’t post it here but can send it via email / anywhere else. This assignment requires both knowledge in Python and excel. Thanks everyone and happy holidays!! submitted by /u/Jealous_League2870 [link] [comments] 

Hi everyone,

I would really need someone’s help with an assignment I have! Can’t post it here but can send it via email / anywhere else. This assignment requires both knowledge in Python and excel.

Thanks everyone and happy holidays!!

submitted by /u/Jealous_League2870
[link] [comments]  Hi everyone, I would really need someone’s help with an assignment I have! Can’t post it here but can send it via email / anywhere else. This assignment requires both knowledge in Python and excel. Thanks everyone and happy holidays!! submitted by /u/Jealous_League2870 [link] [comments]

Read more

List of things to learn in python /u/Helloiamcool101 Python Education

List of things to learn in python /u/Helloiamcool101 Python Education

So it’s my winter vacations and I wanna be good at at least 2 libraries. I’ve started learning selenium but I’m very confused what to learn. I’m doing ai ml. Can someone provide me a list of stuff in selenium that I can learn, along with a suggestion of another library?

submitted by /u/Helloiamcool101
[link] [comments]

​r/learnpython So it’s my winter vacations and I wanna be good at at least 2 libraries. I’ve started learning selenium but I’m very confused what to learn. I’m doing ai ml. Can someone provide me a list of stuff in selenium that I can learn, along with a suggestion of another library? submitted by /u/Helloiamcool101 [link] [comments] 

So it’s my winter vacations and I wanna be good at at least 2 libraries. I’ve started learning selenium but I’m very confused what to learn. I’m doing ai ml. Can someone provide me a list of stuff in selenium that I can learn, along with a suggestion of another library?

submitted by /u/Helloiamcool101
[link] [comments]  So it’s my winter vacations and I wanna be good at at least 2 libraries. I’ve started learning selenium but I’m very confused what to learn. I’m doing ai ml. Can someone provide me a list of stuff in selenium that I can learn, along with a suggestion of another library? submitted by /u/Helloiamcool101 [link] [comments]

Read more

Having problems, specifically with imports not resolved. /u/RegionRelative5890 Python Education

Having problems, specifically with imports not resolved. /u/RegionRelative5890 Python Education

They are called Pip’s I believe. Not totally sure if I need to download them to my computer or if I can copy and paste the Bash into the line of code. I created the program with the help of YouTube but unfortunately I can’t find a video that helps with this problem. Been working at it for a few days now with no resolution lol so anything helps, thanks

submitted by /u/RegionRelative5890
[link] [comments]

​r/learnpython They are called Pip’s I believe. Not totally sure if I need to download them to my computer or if I can copy and paste the Bash into the line of code. I created the program with the help of YouTube but unfortunately I can’t find a video that helps with this problem. Been working at it for a few days now with no resolution lol so anything helps, thanks submitted by /u/RegionRelative5890 [link] [comments] 

They are called Pip’s I believe. Not totally sure if I need to download them to my computer or if I can copy and paste the Bash into the line of code. I created the program with the help of YouTube but unfortunately I can’t find a video that helps with this problem. Been working at it for a few days now with no resolution lol so anything helps, thanks

submitted by /u/RegionRelative5890
[link] [comments]  They are called Pip’s I believe. Not totally sure if I need to download them to my computer or if I can copy and paste the Bash into the line of code. I created the program with the help of YouTube but unfortunately I can’t find a video that helps with this problem. Been working at it for a few days now with no resolution lol so anything helps, thanks submitted by /u/RegionRelative5890 [link] [comments]

Read more

How to learn to apply OOP and SOLID in your code. /u/TheOushen Python Education

How to learn to apply OOP and SOLID in your code. /u/TheOushen Python Education

Hi, I am a beginner Python developer. I plan to work in the backend direction. I learned Python syntax, several frameworks, understood the essence of OOP and SOLID principles, but I ran into a problem that I do not understand how to apply them correctly in practice. Example: having tried to write a simple program (in Tkinter, sqlite3) for registration and authorization, I basically succeeded, but this code looks more like a mess of different functions and strings. Please tell me where I can find material on the principles of writing beautiful code.

submitted by /u/TheOushen
[link] [comments]

​r/learnpython Hi, I am a beginner Python developer. I plan to work in the backend direction. I learned Python syntax, several frameworks, understood the essence of OOP and SOLID principles, but I ran into a problem that I do not understand how to apply them correctly in practice. Example: having tried to write a simple program (in Tkinter, sqlite3) for registration and authorization, I basically succeeded, but this code looks more like a mess of different functions and strings. Please tell me where I can find material on the principles of writing beautiful code. submitted by /u/TheOushen [link] [comments] 

Hi, I am a beginner Python developer. I plan to work in the backend direction. I learned Python syntax, several frameworks, understood the essence of OOP and SOLID principles, but I ran into a problem that I do not understand how to apply them correctly in practice. Example: having tried to write a simple program (in Tkinter, sqlite3) for registration and authorization, I basically succeeded, but this code looks more like a mess of different functions and strings. Please tell me where I can find material on the principles of writing beautiful code.

submitted by /u/TheOushen
[link] [comments]  Hi, I am a beginner Python developer. I plan to work in the backend direction. I learned Python syntax, several frameworks, understood the essence of OOP and SOLID principles, but I ran into a problem that I do not understand how to apply them correctly in practice. Example: having tried to write a simple program (in Tkinter, sqlite3) for registration and authorization, I basically succeeded, but this code looks more like a mess of different functions and strings. Please tell me where I can find material on the principles of writing beautiful code. submitted by /u/TheOushen [link] [comments]

Read more

How do i maintain previously obtained knowledge? /u/Ok-Edge8467 Python Education

How do i maintain previously obtained knowledge? /u/Ok-Edge8467 Python Education

I might not be the best at phrasing it, but you see, my goal is to become a data analyst. I have worked trough python crash course by Eric Matthes (excluding the PyGame and Django blocks) and now going trough an SQL book, the thing is, as I have moved from Python to SQL, I have stopped working at Python and I fear that I will lose experience and knowledge of it, what do I do? Weekly allocate a certain day for revising Python? Solve problems on Leetcode? An advice from someone experienced would be very handy…

submitted by /u/Ok-Edge8467
[link] [comments]

​r/learnpython I might not be the best at phrasing it, but you see, my goal is to become a data analyst. I have worked trough python crash course by Eric Matthes (excluding the PyGame and Django blocks) and now going trough an SQL book, the thing is, as I have moved from Python to SQL, I have stopped working at Python and I fear that I will lose experience and knowledge of it, what do I do? Weekly allocate a certain day for revising Python? Solve problems on Leetcode? An advice from someone experienced would be very handy… submitted by /u/Ok-Edge8467 [link] [comments] 

I might not be the best at phrasing it, but you see, my goal is to become a data analyst. I have worked trough python crash course by Eric Matthes (excluding the PyGame and Django blocks) and now going trough an SQL book, the thing is, as I have moved from Python to SQL, I have stopped working at Python and I fear that I will lose experience and knowledge of it, what do I do? Weekly allocate a certain day for revising Python? Solve problems on Leetcode? An advice from someone experienced would be very handy…

submitted by /u/Ok-Edge8467
[link] [comments]  I might not be the best at phrasing it, but you see, my goal is to become a data analyst. I have worked trough python crash course by Eric Matthes (excluding the PyGame and Django blocks) and now going trough an SQL book, the thing is, as I have moved from Python to SQL, I have stopped working at Python and I fear that I will lose experience and knowledge of it, what do I do? Weekly allocate a certain day for revising Python? Solve problems on Leetcode? An advice from someone experienced would be very handy… submitted by /u/Ok-Edge8467 [link] [comments]

Read more

Online Python Course /u/Astrogirl22709 Python Education

Online Python Course /u/Astrogirl22709 Python Education

I want to learn Python, but I am not in a position to go to college or pay for classes. I need a good/respected free online course that doesn’t take more than a month (There’s a lot of leeway in this regard) and has a certificate of completion. Please give me recommendations, thank you!

submitted by /u/Astrogirl22709
[link] [comments]

​r/learnpython I want to learn Python, but I am not in a position to go to college or pay for classes. I need a good/respected free online course that doesn’t take more than a month (There’s a lot of leeway in this regard) and has a certificate of completion. Please give me recommendations, thank you! submitted by /u/Astrogirl22709 [link] [comments] 

I want to learn Python, but I am not in a position to go to college or pay for classes. I need a good/respected free online course that doesn’t take more than a month (There’s a lot of leeway in this regard) and has a certificate of completion. Please give me recommendations, thank you!

submitted by /u/Astrogirl22709
[link] [comments]  I want to learn Python, but I am not in a position to go to college or pay for classes. I need a good/respected free online course that doesn’t take more than a month (There’s a lot of leeway in this regard) and has a certificate of completion. Please give me recommendations, thank you! submitted by /u/Astrogirl22709 [link] [comments]

Read more

New Spyder 6.0.1 does not have “run selected code option” /u/Environmental-Owl566 Python Education

New Spyder 6.0.1 does not have “run selected code option” /u/Environmental-Owl566 Python Education

So I’m running Spyder 6.0.1 from the latest Anaconda distro. I used to be able to select code and right click on it, then the menu pops up and show “run selected code”. This was present in Spyder 5.5 , but gone in Spyder 6.0. Does anyone know how to fix this?

submitted by /u/Environmental-Owl566
[link] [comments]

​r/learnpython So I’m running Spyder 6.0.1 from the latest Anaconda distro. I used to be able to select code and right click on it, then the menu pops up and show “run selected code”. This was present in Spyder 5.5 , but gone in Spyder 6.0. Does anyone know how to fix this? submitted by /u/Environmental-Owl566 [link] [comments] 

So I’m running Spyder 6.0.1 from the latest Anaconda distro. I used to be able to select code and right click on it, then the menu pops up and show “run selected code”. This was present in Spyder 5.5 , but gone in Spyder 6.0. Does anyone know how to fix this?

submitted by /u/Environmental-Owl566
[link] [comments]  So I’m running Spyder 6.0.1 from the latest Anaconda distro. I used to be able to select code and right click on it, then the menu pops up and show “run selected code”. This was present in Spyder 5.5 , but gone in Spyder 6.0. Does anyone know how to fix this? submitted by /u/Environmental-Owl566 [link] [comments]

Read more