Properly structuring python project /u/bkaterpillar Python Education

I have tried for a long time to understand how to properly structure an pip-installable python project consistently, but I always run into issues. I see so many different ways things are structured, and following tutorials never seems to work well for me.

Overall, I am trying to structure my project like this:

project_name/ ├── .env ├── setup.py ├── src/ │ └── package1/ │ ├── __init__.py │ └── module1.py │ └── package2/ │ ├── __init__.py │ └── module2.py └── tests/ │ ├── test1.py │ ├── test2.py └── scripts/ │ ├── script1.py │ ├── script2.py 

• a src folder that contains all python files intended to be installable. This code is separated into several packages.
• a folder of pytests adjacent to the src folder (to keep them uninstallable)
• a folder of miscellaneous python scripts adjacent to the src folder (to keep them uninstallable)

I want to ensure that: – the package should be installable with the name “project_name”, such that members can be imported like: “from project_name.package1 import module1”. – any code within the src folder can import any other code within the src folder with relative imports. (e.g., module2.py may contain: “from ..package1 import module1”.) – The test scripts and miscellaneous scripts can import from the src code. Preferably, I don’t want to rely on installing the project to do this if I don’t have to. – I don’t want to have to modify sys.path to do any of this. That just seems messy to me. I also don’t want to have to rely on installing the package as non-editable, to reduce steps in my workflow. – The word ‘src’ should not be included in any import paths (it seems that most code I see doesn’t do this).

I have attempted to have a setup.py file with:

from setuptools import setup, find_packages, find_namespace_packages setup( name='project_name', version='0.1', packages=find_packages(where='src'), package_dir={'': 'src'}, install_requires=[], include_package_data=True, ) 

However, installing this in my venv as editable then importing ‘project_name’ gives me a ModuleNotFoundError error, and my ide doesn’t detect the packages.

Other questions: – I sometimes see tests contain an init.py file, and sometimes not. What is the difference and how does it affect how things must be structured/installed? – I sometimes see src itself contain an init.py file, and sometimes I don’t. How would this choice affect setup.py? – I often see a single folder within src which has the intended package name, with all submodules of that package within it, as well as its own init.py file. But it seems redundant to me to have a folder that contains nothing but one other folder, so I thought I’d just flatten that directory into a single ‘src’ directory. Is that sensible or is there some other functionality I am missing? – I sometimes see the tests folder or other adjacent folders have an init.py file, and sometimes I don’t. How does this choice affect the structure of the rest of the project or the install process? – Is it possible to structure the project this way to that using the package in tests or scripts neither required editing sys.path nor installing the project at all?

I have tried for a long time to understand how to properly structure an pip-installable python project consistently, but I always run into issues. I see so many different ways things are structured, and following tutorials never seems to work well for me.

Overall, I am trying to structure my project like this:

project_name/ ├── .env ├── setup.py ├── src/ │ └── package1/ │ ├── __init__.py │ └── module1.py │ └── package2/ │ ├── __init__.py │ └── module2.py └── tests/ │ ├── test1.py │ ├── test2.py └── scripts/ │ ├── script1.py │ ├── script2.py 

• a src folder that contains all python files intended to be installable. This code is separated into several packages.
• a folder of pytests adjacent to the src folder (to keep them uninstallable)
• a folder of miscellaneous python scripts adjacent to the src folder (to keep them uninstallable)

I want to ensure that: – the package should be installable with the name “project_name”, such that members can be imported like: “from project_name.package1 import module1”. – any code within the src folder can import any other code within the src folder with relative imports. (e.g., module2.py may contain: “from ..package1 import module1”.) – The test scripts and miscellaneous scripts can import from the src code. Preferably, I don’t want to rely on installing the project to do this if I don’t have to. – I don’t want to have to modify sys.path to do any of this. That just seems messy to me. I also don’t want to have to rely on installing the package as non-editable, to reduce steps in my workflow. – The word ‘src’ should not be included in any import paths (it seems that most code I see doesn’t do this).

I have attempted to have a setup.py file with:

from setuptools import setup, find_packages, find_namespace_packages setup( name='project_name', version='0.1', packages=find_packages(where='src'), package_dir={'': 'src'}, install_requires=[], include_package_data=True, ) 

However, installing this in my venv as editable then importing ‘project_name’ gives me a ModuleNotFoundError error, and my ide doesn’t detect the packages.

Other questions: – I sometimes see tests contain an init.py file, and sometimes not. What is the difference and how does it affect how things must be structured/installed? – I sometimes see src itself contain an init.py file, and sometimes I don’t. How would this choice affect setup.py? – I often see a single folder within src which has the intended package name, with all submodules of that package within it, as well as its own init.py file. But it seems redundant to me to have a folder that contains nothing but one other folder, so I thought I’d just flatten that directory into a single ‘src’ directory. Is that sensible or is there some other functionality I am missing? – I sometimes see the tests folder or other adjacent folders have an init.py file, and sometimes I don’t. How does this choice affect the structure of the rest of the project or the install process? – Is it possible to structure the project this way to that using the package in tests or scripts neither required editing sys.path nor installing the project at all?