AdamOptimizer

[bpietroiu]

I am trying to get CnnTextClassification to work with IsImportingGraph = false

In AdamOptimizer I found two problems.

1. It does not override Optimizer._prepare so I get a NullReferenceException down the line, when LearningRateTensor is referenced
2. After fixing 1, when running the generated graph, Session.Run returns status : 'var and delta do not have the same shape[41205,128] [6400,128] [[{{node loss/Adam/update_embeddings/ApplyGradientDescent}}]]'

I have no ideea what to do next.

[henon]

When I try to fix errors in TF.NET I do it like this: I run the same program in PyCharm and in VisualStudio and set breakpoints to inspect. Ususally after simultaneously stepping through the code it becomes obvious where they digress.

[Oceania2018]

Can you PR your fixed 1? It will reduce my workload.

[bpietroiu]

Can you PR your fixed 1? It will reduce my workload.

I have created a PR for it- CnnTextClassification debugging #272

[Oceania2018]

@bpietroiu Can you check the latest code?

[bpietroiu]

@bpietroiu Can you check the latest code?

tested with both IsImportingGraph true and false

IsImportingGraph=True
System.Exception: NodeDef mentions attr 'explicit_paddings' not in Op<name=Conv2D; signature=input:T, filter:T -> output:T; attr=T:type,allowed=[DT_HALF, DT_BFLOAT16, DT_FLOAT, DT_DOUBLE]; attr=strides:list(int); attr=use_cudnn_on_gpu:bool,default=true; attr=padding:string,allowed=["SAME", "VALID"]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]; attr=dilations:list(int),default=[1, 1, 1, 1]>; NodeDef: {{node conv2d/Conv2D}}. (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).

IsImportingGraph=False
System.Exception: Operation 'conv2d/Conv2D' has no attr named 'explicit_paddings'.
at Tensorflow.Status.Check(Boolean throwException) in D:\sandbox\TensorFlow.NET\src\TensorFlowNET.Core\Status\Status.cs:line 48
at Tensorflow.Operation.get_attr(String name) in D:\sandbox\TensorFlow.NET\src\TensorFlowNET.Core\Operations\Operation.cs:line 228
at Tensorflow.Gradients.nn_grad._Conv2DGrad(Operation op, Tensor[] grads) in D:\sandbox\TensorFlow.NET\src\TensorFlowNET.Core\Gradients\nn_grad.cs:line 112
--- End of inner exception stack trace ---

On a side-note, something is happening with tensorflow.dll, in tensorflowlib\runtimes\win-x64\native i have a zero bytes file.

[Oceania2018]

@bpietroiu Please replace tensorflow.dll with v1.14 https://github.com/SciSharp/TensorFlow.NET/tree/master/tensorflowlib

[bpietroiu]

@bpietroiu Please replace tensorflow.dll with v1.14 https://github.com/SciSharp/TensorFlow.NET/tree/master/tensorflowlib

tested with both IsImportingGraph true and false, NUM_EPOCHS=10

IsImportingGraph=true
loss: 0.39, Validation Accuracy: 0.94

IsImportingGraph=false
loss: 2.14, Validation Accuracy: 0.65

The generated graph does not perform just as good as the imported one
I will debug graph construction, side by side with the python one, to see where is the difference.

are there any tools to compare two untrained graphs?

[Oceania2018]

I usually use JsonConvert to export the graph. You can also export and import to python, and check the graph https://github.com/SciSharp/TensorFlow.NET/wiki/Debugging.

[Oceania2018]

@bpietroiu Find any clue?