scy,
@scy@chaos.social avatar

Imports in Python can be confusing.

I just saw someone ask "why do I have to prepend a dot to import a file in the same directory?"

That depends on whether the file with the import statement is in a package or not.

But, whether Python considers it to be in a package depends on how you imported (or ran) that file. You can't determine it from the file's content or the filesystem structure!

Check alt text (image description) for explanations of the examples.

#Python #import

Running four commands from the parent directory. "python3 mypkg/abs_import.py" prints "in pkg", because Python found mymod.py in the same directory as the file it has been asked to run. That's because sys.path (which is used to define search order for modules) is initialized like this (quoting from the docs): "The first entry in the module search path is the directory that contains the input script, if there is one. Otherwise, the first entry is the current directory, which is the case when executing the interactive shell, a -c command, or -m module." "python3 mypkg/rel_import.py" throws an ImportError: "attempted relative import with no known parent package". Just because you're running a file in a subdirectory doesn't make this directory a package. "python3 -m mypkg.rel_import" prints "in pkg", because Python is now interpreting mypkg as a package name, has found the rel_import module in that package, and is able to do relative imports from there. "python3 -m mypkg.abs_import" prints "top level". Remember the documentation from above: If there is no input script (and there isn't, we're asking Python to resolve and run a module instead), sys.path will first look in the current directory, i.e. the one containing the mypkg package, because that's the one we're currently in.
We now change into the mypkg directory with a "cd" command. "python3 abs_import.py" prints "in pkg", because Python is going to look for "mymod" in the directory containing the input script (which happens to be the current directory, but that's not relevant). "python3 rel_import.py" throws an ImportError "attempted relative import with no known parent package" again. Understandably, because Python has no way of knowing that the directory we're currently in can be interpreted as a package. "python3 -m abs_import" prints "in pkg", because sys.path first looks for mymod the current directory. "python3 -m rel_import" raises an ImportError "attempted relative import with no known parent package" again. That's because, in contrast to what we did in the last example in the previous screenshot, we're now just using "rel_import" as the module name we're asking Python to run, without the "mypkg" prefix. Adding the prefix wouldn't work, because our current directory is already inside the mypkg package and Python (correctly) wouldn't find another "mypkg" directory in it. But without the prefix, Python doesn't know that the "rel_import" module resides in a package at all.
We now move into the parent directory again ("cd ..") and delete the top-level mymod.py file. Then, we attempt the examples from the second image again. "python3 mypkg/abs_import.py" prints "in pkg" as before, because Python found mymod.py in the same directory as the file it has been asked to run. "python3 mypkg/rel_import.py" throws an ImportError: "attempted relative import with no known parent package", just like before, because it interprets the file path as a script to run, not as a module in a package. "python3 -m mypkg.rel_import" prints "in pkg", just like before, because Python is interpreting mypkg as a package name, has found the rel_import module in that package, and is able to do relative imports from there. "python3 -m mypkg.abs_import" throws a ModuleNotFoundError "no module named 'mymod'. Before, it printed "top level", but now we have deleted the top-level mymod.py file that it was importing.

uint8_t,
@uint8_t@chaos.social avatar

@scy then import system is honestly the most confusing and worst part of Python

I dread it every time even tho I love using Python, and I’m doing it daily

scy,
@scy@chaos.social avatar

Also, while in Python 2 packages had to have an init.py file (which is what made them a package), this is no longer true for Python 3. init.py files are completely optional.

An init.py file is being interpreted the first time some module from your package is imported. But it's also what allows you to import (from) a package instead of a module.

Many packages use this to provide easier access to things defined in sub-modules, which can be extra confusing for inexperienced devs.

Five example commands, all running from the top directory. "python3 mod_import.py" prints "

veronica,
@veronica@mastodon.online avatar

@scy Even if they were optional, packaging tools still tend to expect them for auto-discovery. We've run into this a few times at work. I usually drop them except for in the root folder of a package.

scy,
@scy@chaos.social avatar

@veronica Good to know, thanks!

I don't really understand why you're making an exception for the root folder though…?

veronica,
@veronica@mastodon.online avatar

@scy Several reasons. For libraries, I use it to control how things are imported by the user by exposing it there. Which means I can restructure the files behind it without changing the import path.

For applications, I use it as the entry point.

In both cases I often do some initialisation there, like set up logging, etc.

scy,
@scy@chaos.social avatar

Damn, "mod_import.py" and "rel_mod_import.py" should have been called "pkg_import.py" and "rel_pkg_import.py" because they're importing the package itself (which is basically equivalent to importing the init.py) instead of a module.

I hope it's okay that I'm not redoing the screenshots and descriptions, I've spend way too much time on these two posts already.

scy,
@scy@chaos.social avatar

Also, remember how I said whether you need to prepend a dot or not depends on whether you're in a package or not?

That's not entirely correct. It depends on your search path. If you run "python3 -m mypkg.abs_import", you're in the parent directory of mypkg, and Python modules inside of it won't be found without prefixing "mypkg." or doing a relative import.

Except … if you modify the search path, e.g. by prepending the "mypkg" directory to it, for example using the PYTHONPATH env variable.

traumaphoenix,
@traumaphoenix@chaos.social avatar

@scy and then you have packages importing other packages and people will do:

from your_package import magic_internal_code

because

your_package/init.py

from ._internal import magic_internal_code

🦋

traumaphoenix,
@traumaphoenix@chaos.social avatar

@scy this is still the most innocent “couldn’t have known” form, i’ve also seen people proxy internals through .main

scy,
@scy@chaos.social avatar

@traumaphoenix Ah, good point, I should probably add a short explanation, too. Thanks!

rfnix,
@rfnix@piaille.fr avatar

@scy Yeah, I think this is the number one thing I've seen people who are relatively new to Python struggle with when they start to interact with a somewhat larger project. Imports really are super confusing...

(I personally ended up converging into two main import methodologies for simplicity... (1) is always use absolute from a single root directory and (2) use absolute, but mangle sys.path as needed at the top of the file before the imports. I generally use (1) for proper code and (2) for quick and dirty experiments... but it's not a good way to do Python at all)

scy,
@scy@chaos.social avatar

@rfnix Since sys.path is global for the whole application, I'd be pretty furious if a random library I'm importing would mess with it just to fix their imports.

I don't think that's what you're suggesting, but I just wanted to make it extra clear that messing with sys.path in random files in your project is basically asking for trouble.

  • All
  • Subscribed
  • Moderated
  • Favorites
  • python
  • rosin
  • Youngstown
  • osvaldo12
  • khanakhh
  • slotface
  • tacticalgear
  • InstantRegret
  • ngwrru68w68
  • kavyap
  • DreamBathrooms
  • thenastyranch
  • everett
  • magazineikmin
  • Durango
  • provamag3
  • GTA5RPClips
  • ethstaker
  • modclub
  • mdbf
  • cisconetworking
  • Leos
  • normalnudes
  • cubers
  • megavids
  • tester
  • anitta
  • JUstTest
  • lostlight
  • All magazines