Skip to content

Conversion from Python datetime to Julia DateTime fails #265

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
luc-j-bourhis opened this issue Feb 5, 2023 · 6 comments
Open

Conversion from Python datetime to Julia DateTime fails #265

luc-j-bourhis opened this issue Feb 5, 2023 · 6 comments

Comments

@luc-j-bourhis
Copy link

luc-j-bourhis commented Feb 5, 2023

using CondaPkg
using PythonCall
using Dates
datetime = pyimport("datetime")
t = datetime.datetime.now()
pyconvert(DateTime, t)

results in

cannot convert this Python 'datetime' to a Julia 'DateTime'

Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] macro expansion
   @ ~/.julia/packages/PythonCall/3GRYN/src/convert.jl:355 [inlined]
 [3] macro expansion
   @ ~/.julia/packages/PythonCall/3GRYN/src/Py.jl:131 [inlined]
 [4] pyconvert(#unused#::Type{DateTime}, x::Py)
   @ PythonCall ~/.julia/packages/PythonCall/3GRYN/src/convert.jl:370

with the minimalist CondaPkg.toml

[deps]
python = "3.9"

but it fails with Python 3.10 and 3.11 too.

This is with CondaPkg v0.2.17 and PythonCall v0.9.10

@cjdoris
Copy link
Collaborator

cjdoris commented Feb 9, 2023

Hmm I think this is because a Julia DateTime is to millisecond precision whereas a Python datetime is to microsecond precision, and PythonCall currently refuses to do the conversion unless the microseconds are zero.

I could add a conversion rule so that if you specifically ask for a DateTime then it will do the lossy conversion.

@luc-j-bourhis
Copy link
Author

Such a conversion would work fine for what I am doing. Thanks.

@cjdoris
Copy link
Collaborator

cjdoris commented Feb 10, 2023

In the meantime, you can do this to get the time to the nearest second:

using CondaPkg
using PythonCall
using Dates
datetime = pyimport("datetime")
t = datetime.datetime.now()
# t = t.replace(microsecond=0)  # nearest second
t = t.replace(microsecond=(t.microsecond//1000)*1000)
pyconvert(DateTime, t)

@github-actions
Copy link
Contributor

github-actions bot commented Sep 1, 2023

This issue has been marked as stale because it has been open for 30 days with no activity. If the issue is still relevant then please leave a comment, or else it will be closed in 7 days.

@github-actions github-actions bot added the stale Issues about to be auto-closed label Sep 1, 2023
@luc-j-bourhis
Copy link
Author

luc-j-bourhis commented Sep 1, 2023

Your proposition gives me a stacktrace:

MethodError: no method matching //(::Py, ::Int64)

Closest candidates are:
  //(!Matched::Integer, ::Integer)
   @ Base rational.jl:62
  //(!Matched::Rational, ::Integer)
   @ Base rational.jl:64
  //(!Matched::Complex, ::Real)
   @ Base rational.jl:78

I tried to add pyconvert(t.microsecond, Int) but that does not work either

MethodError: no method matching pyconvert(::Py, ::Type{Int64})

Closest candidates are:
  pyconvert(!Matched::Type{T}, ::Any) where T
   @ PythonCall ~/.julia/packages/PythonCall/qTEA1/src/convert.jl:370
  pyconvert(!Matched::Type{T}, ::Any, !Matched::Any) where T
   @ PythonCall ~/.julia/packages/PythonCall/qTEA1/src/convert.jl:371

But honestly, converting a Python datetime.datetime to DateTime is mundane enough that it should work out of the box. The truncation from microseconds to milliseconds is unavoidable, and the programmer who needs to do that conversion will bite the bullet.

Note: this is now with PythonCall 0.9.14

@github-actions github-actions bot removed the stale Issues about to be auto-closed label Sep 2, 2023
@cjdoris
Copy link
Collaborator

cjdoris commented Sep 13, 2023

It's pyconvert(Int, t.microsecond).

The pyconvert rules are lossless by default, hence refusing to truncate. However there's an argument that small losses in precision are OK (just as convert(Float16, 1.2) loses precision) so maybe it's OK in this case. I've just been burned too many times by Python happily converting floats to ints.

In the future, I'm planning to add keyword args to pyconvert to allow more flexible conversion, so that e.g. pyconvert(DateTime, t; truncate=true) will allow a loss of precision.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants