Error using podman cp on Windows

[PlaidPhantom]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

I have a fresh install of podman on Windows 10 & WSL2 with a couple of containers installed and running, and I'm getting a strange error message when I try to copy a file into a container using podman cp:

C:\Users\myuser>cd C:\location\of\file\

C:\location\of\file\>podman cp myfile.bak mycontainer:/var/opt/backups
error rewriting "C:\\location\\of\\file\\myfile.bak" to be relative to "/": expected root directory to be an absolute path, got "/"

Additional information you deem important (e.g. issue happens only occasionally):

FWIW I'll note that I initially attempted this using an MSYS2 shell, but I'm fairly confident that's not an issue as I've also gotten this error through both Command Prompt and Powershell. The above example was pulled from Command Prompt (and then anonymized, of course).

Output of podman version:

Client:       Podman Engine
Version:      4.1.1
API Version:  4.1.1
Go Version:   go1.16.15
Git Commit:   f73d8f8875c2be7cd2049094c29aff90b1150241
Built:        Wed Jun 15 08:17:12 2022
OS/Arch:      windows/amd64

Server:       Podman Engine
Version:      4.1.1
API Version:  4.1.1
Go Version:   go1.16.15
Built:        Wed Jun 15 09:32:06 2022
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.26.1
  cgroupControllers: []
  cgroupManager: cgroupfs
  cgroupVersion: v1
  conmon:
    package: conmon-2.1.1-2.fc35.x86_64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.1, commit: '
  cpuUtilization:
    idlePercent: 99.65
    systemPercent: 0.13
    userPercent: 0.21
  cpus: 8
  distribution:
    distribution: fedora
    variant: container
    version: "35"
  eventLogger: file
  hostname: pp-907-11538
  idMappings:
    gidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
    uidmap:
    - container_id: 0
      host_id: 1000
      size: 1
    - container_id: 1
      host_id: 100000
      size: 65536
  kernel: 5.10.16.3-microsoft-standard-WSL2
  linkmode: dynamic
  logDriver: journald
  memFree: 21931720704
  memTotal: 26743394304
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun-1.4.5-2.fc35.x86_64
    path: /usr/bin/crun
    version: |-
      crun version 1.4.5
      commit: c381048530aa750495cf502ddb7181f2ded5b400
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/user/1000/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: true
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: true
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: slirp4netns-1.1.12-2.fc35.x86_64
    version: |-
      slirp4netns version 1.1.12
      commit: 7a104a101aa3278a2152351a082a6df71f57c9a3
      libslirp: 4.6.1
      SLIRP_CONFIG_VERSION_MAX: 3
      libseccomp: 2.5.3
  swapFree: 7516192768
  swapTotal: 7516192768
  uptime: 2h 47m 23.29s (Approximately 0.08 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries:
  search:
  - registry.fedoraproject.org
  - registry.access.redhat.com
  - docker.io
  - quay.io
store:
  configFile: /home/user/.config/containers/storage.conf
  containerStore:
    number: 2
    paused: 0
    running: 2
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /home/user/.local/share/containers/storage
  graphRootAllocated: 269490393088
  graphRootUsed: 3479621632
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/user/1000/containers
  volumePath: /home/user/.local/share/containers/storage/volumes
version:
  APIVersion: 4.1.1
  Built: 1655303526
  BuiltTime: Wed Jun 15 09:32:06 2022
  GitCommit: ""
  GoVersion: go1.16.15
  Os: linux
  OsArch: linux/amd64
  Version: 4.1.1

Package info (e.g. output of rpm -q podman or apt list podman):

New install via podman-v4.1.1.msi

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/main/troubleshooting.md)

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

As stated, Windows 10 / WSL 2

[mheon]

@n1hility PTAL

[n1hility]

Hi @PlaidPhantom I'll put this on my TODO to add support for. In the meantime as a workaround you can access your windows drive through the mount that is on the wsl linux host, using unix path conventions, like this:

podman cp /mnt/c/users/foo/bar/file.bak mycontainer:/var/opt/backups

[PlaidPhantom]

Thanks that worked for my need right now.

[wernerk-bbd]

@n1hility I can't get the workaround to work.
In the podman wsl container if I run
podman cp /mnt/c/temp/file mycontainer:/app
it doesn't know about the container
if I try
'podman.exe cp /mnt/c/temp/file mycontainer:/app`
It gives an access denied message to the file on windows

[dancohen21]

@n1hility
PS D:\testing> podman cp /mnt/d/testing/1130_symapi_db.bin se92:/root
Error: "/mnt/d/testing/1130_symapi_db.bin" could not be found on the host: No such file or directory

[jakubvanecek]

Hi @dancohen21,

I tried to run container (new one) inside wsl and it worked.

[dancohen21]

Hi @dancohen21,

I tried to run container (new one) inside wsl and it worked.

I'm running Win 10 Enterprise 20H2. upgraded to the v4.2.0 release. installed with the podman-v4.2.0.msi windows MSI. I couldn't be more happy with the direction here with the Remote tools executing in WSL container. I'm building a local container from opensuse/leap. which has worked well in the past. I'm having issues with 'podman cp' command. Also, I've discovered that 'bash -c' works well, but 'wsl -d <wsl_machine> --exec' does not work at all running podman commands (stepping into the wsl machine and executing podman commands).

I found a series of commands to support my workflow at the moment.
copy .<file> \wsl$\podman-machine-default\root
bash -c "podman mount se92"
bash -c "cp /root/ /root/"
bash -c "podman unmount se92"

However after getting everything working, its running well for ~30 seconds and starts crawling. Which is very unexpected. On a prior machine using wsl + podman (without the windows installer) I didn't have any issues with performance. In fact, the execution in the podman container was faster than in native windows (python app).

I'll wait for a few releases and then give it another try from scratch. Seems like there is active development / improvements. Very grateful to everyone working on these projects.

[n1hility]

@dancohen21 Sorry for the delay on fixing cp, I'll bump it up higher on my queue. BUT, if there are problems accessing the drive that would be problematic for a fix there to work. if you enter the wsl host using wsl, do you see a /mnt/c but not a /mnt/d ? Does it help if you do a forced shutdown:

podman machine stop
wsl --shutdown
podman machine start

Regarding running direct commands from WSL, you need to use the enterns shell wrapper. This is because running systemd on WSL requires a nested namespace, and while ssh enters you appropriately, wsl still connects you into the outer namespace. There is a profile script that runs and auto-enters you, but this is bypassed by direct commands.

Here is some examples:

Rootful

wsl -d podman-machine-default -u root enterns podman info

Rootless

wsl -d podman-machine-default -u user enterns podman info

[dancohen21]

@n1hility super helpful. Thanks for the clarification. I will experiment with the enterns shell wrapper. My googling did not uncover this. I see where this is documented now: https://github.com/containers/podman/blob/main/docs/tutorials/podman-for-windows.md
Looks like I have more to read up on. I didn't realize how much work has been done to enable systemd with wsl.
Yes, the /mnt/c and /mnt/d mounts are all in place automatically.
wsl -d podman-machine-default enterns ls /mnt
shows c, d, and wsl directories. and I can via windows terminal get a shell into the wsl machine and navigate/access the mounted file systems as expected. It does seem that accessing the drives is not a problem.
my .wslconfig file is simple
[wsl2]
memory=16GB
processors=4
please set feature priorities as you see fit. I'm not contributing to the coding.

any thoughts on how I can track down why my python app starts crawling in the podman container but runs normally in native python in windows? and if this is out of scope, no worries.

[n1hility]

@dancohen21 ok interesting if the mounts are there that cp command should then work from windows. I'm not yet sure why it's not. Does it work from the interactive wsl prompt ( vs the direct command - e.g. wsl -d podman-machine default -u user for rootless and wsl -d podman-machine default -u root for rootful)?

As to performance, is there any chance that python app is doing a lot of FS activity against a windows drive path? The bridge layer over plan 9 is not as fast as the virtual ext partition used by the local linux FS. If you need to monitor a win location, one example case where plan 9 can struggle is directories with a large number of entries. If you have a temp folder that has lots of churn that could explain why it was fast before and slower now.

[dancohen21]

@n1hility
I was able to get a file copy into my container working with the following.
I redeployed my podman machine, rebuilt my container...
this does work:
wsl -d podman-machine-default -u user enterns podman cp /mnt/d/2022_Projects/assessments/HK197500195.bin se92:/root/

[dancohen21]

@n1hility
Just installed 4.3.0 on windows [with WSL]
my laptop has a "c" drive and a "d" drive
I'm trying to copy a file from my "d" drive to my container "se92"

this command works. It runs from the context of the podman-machine-default.
wsl -d podman-machine-default -u user enterns podman cp /mnt/d/2022_Projects/CCCC/1861_symapi_db_20221021.bin se92:/root/

this command is not working. running from the remote podman tools in windows proper context
podman cp 'D:\2022_Projects\CCCC\1272_symapi_db_20221021.bin' se92:/root/test.bin
Error: container "D" does not exist

this command is not working either:
podman cp .\1272_symapi_db_20221021.bin se92:/root/test.bin
error rewriting "D:\2022_Projects\CCCC\1272_symapi_db_20221021.bin" to be relative to "/": expected root directory to be an absolute path, got "/"

nor
podman cp /mnt/d/2022_Projects/CCCC/1272_symapi_db_20221021.bin se92:/root/test.bin

Is there a syntax I'm missing?
the in-context help says:
You can copy from the container's file system to the local machine or the reverse, from the local filesystem to the container.

[miguele]

We have podman desktop version 4.3.0 and have this error " error rewriting "C:\XXX\XXX" to be relative to "/": expected root directory to be an absolute path, got "/" ".
I have been execute this code inside .BAT in windows :

set URL_SCRIPTS=/XXX/XXX

podman run --name example-postgresql -d -p 5432:5432 -e POSTGRES_PASSWORD=123456 -e POSTGRES_USER=user postgres:13

cd "%URL_SCRIPTS%"
podman exec example-postgresql mkdir /sql
podman cp "%URL_SCRIPTS%" example-postgresql:/sql/

How can i do to make it work?

[dancohen21]

@miguele
I believe this will work.
wsl -d podman-machine-default -u user enterns podman cp /mnt/c/"your file location" container:/sql/

[miguele]

@dancohen21
I put in .BAT:

set URL_SCRIPTS=/XXX/XXX

podman run --name example-postgresql -d -p 5432:5432 -e POSTGRES_PASSWORD=123456 -e POSTGRES_USER=user postgres:13

cd "%URL_SCRIPTS%"

podman exec example-postgresql mkdir /sql
wsl -d podman-machine-default -u user enterns podman cp /mnt/c/%URL_SCRIPTS% example-postgresql:/sql/


podman exec example-postgresql psql user user -f /sql/001.CREATE_BASE.sql

Apparently it copies the files because there is no error, but when on the next line it tries to run the 001.CREATE_BASE.sql script that it should have copied, it can't find it. It doesn't seem to fail, but it doesn't copy the content. I've checked the path and I'm sure the script is there

[mibadim]

"Prefixing" docker cp with wsl -d podman-machine-default -u user enterns also worked for me. Concretely, I wrote

wsl -d podman-machine-default -u user enterns podman cp "myfile.txt" my-container-name:/tmp/

[J-qak]

Simple workaround for me is to use podman machine ssh command like this:

podman machine ssh "podman cp /mnt/c/dev/file.py container:/usr/src/app"

[swizardlv]

why not fix it instead of using any workaround??

[rhatdan]

PRs welcome.

[dgendill]

I've been using the wsl copy workaround for a while now but I've recently upgraded to Podman Desktop v1.15.0 and Podman v5.3.1 and for some reason it's stopped worked and I get the error "Error: container "sql_server_2017" does not exist"

wsl -d podman-machine-default -u user enterns podman cp /mnt/c/Users/dgendill/Documents/file.sql sql_server_2017:/home/file.sql

I'm certain I have the "sql_server_2017" container running because I can see it running in Podman Desktop.

I also tried running podman cp /mnt/c/Users/dgendill/Documents/file.sql sql_server_2017:/home/file.sql and I get a message saying the file doesn't exist, but I'm certain that path exists.

Error: "/mnt/c/Users/dgendill/Documents/file.sql" could not be found on the host: no such file or directory

Not exactly sure what to try next or how to debug this.

---

Edit: I was able to get everything working by...

1. Ran Cleanup / Purge data in Troubleshooting > Repair
2. Ran podman machine init
3. Ran podman machine start
4. Ran podman pull mcr.microsoft.com/mssql/server:2017-latest
5. Ran podman run -d --name sql_server_2017 ...

Then I was able to run the wsl command in windows cmd. Running in windows bash didn't work.

[ravindUwU]

I think this issue makes it pretty much impossible to copy files over to the container if WSL (which allows the /mnt/c/... paths mentioned in the workarounds above) is not enabled. The COPY directive of a Containerfile seems to work just fine, though 🤔.

[abrxu]

any news @n1hility ?

[wernerk-bbd]

Great thanks

[akalankapagoda]

After all the workarounds suggested here failed, I simply hosted my files using a simple python http server.
python -m http.server 8000

And wget from inside the container.