Console log: 'Microsoft.ML.TensorFlow.Tests' from job b21a52b7-f329-47b1-85e5-fdd0c47420c7 workitem cacbfaab-4f4c-461e-9097-853f2b118263 (windows.10.amd64.open) executed on machine a00F8EK running Windows-2016Server-10.0.14393-SP0 C:\h\w\AAB00940\w\C4F409A1\e>set ML_TEST_DATADIR=C:\h\w\AAB00940\p C:\h\w\AAB00940\w\C4F409A1\e>set MICROSOFTML_RESOURCE_PATH=C:\h\w\AAB00940\w\C4F409A1\e C:\h\w\AAB00940\w\C4F409A1\e>set PATH=C:\h\w\AAB00940\p\dotnet-cli;C:\python3\Scripts\;C:\python3\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Debuggers\x64;C:\Program Files\Microsoft SQL Server\160\Tools\Binn\;C:\Users\runner\AppData\Local\Microsoft\WindowsApps C:\h\w\AAB00940\w\C4F409A1\e>call .\runTests.cmd ----- start Wed 09/16/2026 16:46:37.14 =============== To repro directly: ===================================================== pushd C:\h\w\AAB00940\w\C4F409A1\e\ C:\h\w\AAB00940\p/xunit-runner/tools/net48/xunit.console.exe Microsoft.ML.TensorFlow.Tests.dll -notrait Category=SkipInCI -xml testResults.xml popd =========================================================================================================== C:\h\w\AAB00940\w\C4F409A1\e>C:\h\w\AAB00940\p/xunit-runner/tools/net48/xunit.console.exe Microsoft.ML.TensorFlow.Tests.dll -notrait Category=SkipInCI -xml testResults.xml xUnit.net Console Runner v2.9.3+9712244020 (64-bit .NET Framework 4.8, runtime: 4.0.30319.42000) Discovering: Microsoft.ML.TensorFlow.Tests 2026-09-16 16:46:42.568283: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.626766: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.704143: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.765828: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.823985: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.880750: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.938213: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:42.994603: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.052628: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.109142: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.166128: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.224644: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.299667: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.379983: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.454592: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.511517: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.567051: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.623550: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.679221: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.735628: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.791828: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.847390: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.903283: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:43.960218: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.016714: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.072294: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.127722: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.183224: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.239515: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.296004: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.355258: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.414914: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.474291: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:44.532903: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. Discovered: Microsoft.ML.TensorFlow.Tests Starting: Microsoft.ML.TensorFlow.Tests Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputShapeTest 2026-09-16 16:46:47.725425: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-16 16:46:47.745106: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_shape_test 2026-09-16 16:46:47.745790: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:46:47.745957: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_shape_test 2026-09-16 16:46:47.747384: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2026-09-16 16:46:47.748885: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled 2026-09-16 16:46:47.750104: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 4556 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputShapeTest with memory usage 615,538,688.00 and max memory usage 616,390,656.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault Saver not created because there are no variables in the graph to restore Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\sw53fvye.u2g\custom_retrained_model_based_on_resnet_v2_50_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault with memory usage 987,127,808.00 and max memory usage 2,623,062,016.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithExponentialLRScheduling Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 0, Accuracy: 0.56, Cross-Entropy: 1.318978, Learning Rate: 0.01 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.5635227 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 1, Accuracy: 0.82, Cross-Entropy: 0.6796401, Learning Rate: 0.0094 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.3488373 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 2, Accuracy: 0.98, Cross-Entropy: 0.343706, Learning Rate: 0.0094 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.2695257 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.248996, Learning Rate: 0.008836 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.2307002 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.1940439, Learning Rate: 0.008836 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.2061126 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1608586, Learning Rate: 0.008305839 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1897818 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1370995, Learning Rate: 0.008305839 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.177311 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1205368, Learning Rate: 0.007807489 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1679253 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1074084, Learning Rate: 0.007807489 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1601414 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.09753735, Learning Rate: 0.00733904 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1539136 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.08923988, Learning Rate: 0.00733904 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1485151 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.08270638, Learning Rate: 0.006898697 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1440345 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.07700094, Learning Rate: 0.006898697 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1400428 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.07236744, Learning Rate: 0.006484775 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1366466 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.0682116, Learning Rate: 0.006484775 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1335641 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.06476115, Learning Rate: 0.006095689 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1308938 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.06160481, Learning Rate: 0.006095689 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.1284372 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.05894057, Learning Rate: 0.005729948 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1262796 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.0564662, Learning Rate: 0.005729948 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1242744 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.05435076, Learning Rate: 0.005386151 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.1224939 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.05236237, Learning Rate: 0.005386151 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1208258 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.05064506, Learning Rate: 0.005062982 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1193316 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.04901516, Learning Rate: 0.005062982 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.1179228 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.04759575, Learning Rate: 0.004759203 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.1166517 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.04623774, Learning Rate: 0.004759203 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.1154468 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.04504711, Learning Rate: 0.004473651 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.1143531 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 26, Accuracy: 1, Cross-Entropy: 0.04390027, Learning Rate: 0.004473651 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 26, Accuracy: 1, Cross-Entropy: 0.1133119 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 27, Accuracy: 1, Cross-Entropy: 0.04288894, Learning Rate: 0.004205232 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 27, Accuracy: 1, Cross-Entropy: 0.1123618 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 28, Accuracy: 1, Cross-Entropy: 0.0419093, Learning Rate: 0.004205232 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 28, Accuracy: 1, Cross-Entropy: 0.1114539 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 29, Accuracy: 1, Cross-Entropy: 0.04104123, Learning Rate: 0.003952918 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 29, Accuracy: 1, Cross-Entropy: 0.1106217 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 30, Accuracy: 1, Cross-Entropy: 0.04019617, Learning Rate: 0.003952918 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 30, Accuracy: 1, Cross-Entropy: 0.1098242 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 31, Accuracy: 1, Cross-Entropy: 0.03944424, Learning Rate: 0.003715743 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 31, Accuracy: 1, Cross-Entropy: 0.1090903 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 32, Accuracy: 1, Cross-Entropy: 0.03870922, Learning Rate: 0.003715743 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 32, Accuracy: 1, Cross-Entropy: 0.108385 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 33, Accuracy: 1, Cross-Entropy: 0.03805279, Learning Rate: 0.003492798 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 33, Accuracy: 1, Cross-Entropy: 0.1077338 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 34, Accuracy: 1, Cross-Entropy: 0.03740868, Learning Rate: 0.003492798 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 34, Accuracy: 1, Cross-Entropy: 0.1071064 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 35, Accuracy: 1, Cross-Entropy: 0.03683168, Learning Rate: 0.00328323 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 35, Accuracy: 1, Cross-Entropy: 0.1065256 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 36, Accuracy: 1, Cross-Entropy: 0.03626369, Learning Rate: 0.00328323 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 36, Accuracy: 1, Cross-Entropy: 0.1059648 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 37, Accuracy: 1, Cross-Entropy: 0.03575343, Learning Rate: 0.003086236 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 37, Accuracy: 1, Cross-Entropy: 0.1054442 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 38, Accuracy: 1, Cross-Entropy: 0.03524971, Learning Rate: 0.003086236 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 38, Accuracy: 1, Cross-Entropy: 0.1049407 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 39, Accuracy: 1, Cross-Entropy: 0.03479608, Learning Rate: 0.002901062 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 39, Accuracy: 1, Cross-Entropy: 0.1044722 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 40, Accuracy: 1, Cross-Entropy: 0.03434711, Learning Rate: 0.002901062 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 40, Accuracy: 1, Cross-Entropy: 0.1040185 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 41, Accuracy: 1, Cross-Entropy: 0.03394192, Learning Rate: 0.002726999 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 41, Accuracy: 1, Cross-Entropy: 0.1035953 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 42, Accuracy: 1, Cross-Entropy: 0.03353996, Learning Rate: 0.002726999 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 42, Accuracy: 1, Cross-Entropy: 0.1031849 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 43, Accuracy: 1, Cross-Entropy: 0.03317649, Learning Rate: 0.002563379 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 43, Accuracy: 1, Cross-Entropy: 0.1028014 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 44, Accuracy: 1, Cross-Entropy: 0.03281517, Learning Rate: 0.002563379 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 44, Accuracy: 1, Cross-Entropy: 0.1024289 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 45, Accuracy: 1, Cross-Entropy: 0.03248792, Learning Rate: 0.002409576 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 45, Accuracy: 1, Cross-Entropy: 0.1020804 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 46, Accuracy: 1, Cross-Entropy: 0.03216198, Learning Rate: 0.002409576 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 46, Accuracy: 1, Cross-Entropy: 0.1017415 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 47, Accuracy: 1, Cross-Entropy: 0.03186632, Learning Rate: 0.002265001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 47, Accuracy: 1, Cross-Entropy: 0.1014239 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 48, Accuracy: 1, Cross-Entropy: 0.03157135, Learning Rate: 0.002265001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 48, Accuracy: 1, Cross-Entropy: 0.1011147 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 49, Accuracy: 1, Cross-Entropy: 0.03130336, Learning Rate: 0.002129101 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 49, Accuracy: 1, Cross-Entropy: 0.1008245 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithExponentialLRScheduling with memory usage 1,507,880,960.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifar Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifar with memory usage 1,523,662,848.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransforCifarEndToEndTest2 Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransforCifarEndToEndTest2 with memory usage 1,706,999,808.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationBadImages Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformObjectDetectionTest [SKIP] Model files are not available yet Saver not created because there are no variables in the graph to restore 2026-09-16 16:50:02.325756: W tensorflow/core/framework/local_rendezvous.cc:404] Local rendezvous is aborting with status: INVALID_ARGUMENT: Trying to decode BMP format using a wrong op. Use `decode_bmp` or `decode_image` instead. Op used: DecodeJpeg [[{{node DecodeJpeg}}]] Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbspm353.gho\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationBadImages with memory usage 1,538,768,896.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTest with memory usage 1,558,921,216.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorflowPlaceholderShapeInferenceTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorflowPlaceholderShapeInferenceTest with memory usage 1,559,216,128.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.5614241 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.3491761 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.2698055 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.2309191 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.2062127 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1897748 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1772215 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1677665 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1599289 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1536536 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.148217 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1437017 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1396813 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1362582 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1331531 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1304613 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.1279864 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1258112 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1237908 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.1219956 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1203147 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1188081 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.1173883 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.1161064 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 1,553,580,032.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.5484064 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 1, Cross-Entropy: 0.3861168 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.3264058 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.289847 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.2654631 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.2473012 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.2335936 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.2224744 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.213547 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.2059433 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1996044 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1940314 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1892669 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1849844 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1812567 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1778512 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.174847 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1720683 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1695916 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.1672785 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1651998 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1632434 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_2\custom_retrained_model_based_on_mobilenet_v2.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 1,806,622,720.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.6525684 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.4046155 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.3154579 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 0.8888889, Cross-Entropy: 0.2750051 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 0.8888889, Cross-Entropy: 0.2492247 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.2322816 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.2191532 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.2092276 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.2009111 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1941963 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1883548 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1834499 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.179089 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1753279 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1719359 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1689521 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.1662336 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1638055 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1615764 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.159561 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1576999 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1560004 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.1544234 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.1529717 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.1516193 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.1503656 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_3\custom_retrained_model_based_on_resnet_v2_50_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 1,610,854,400.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.6731744 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.5248026 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.4498506 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.4066119 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.37776 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.3571833 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.3413683 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.3288983 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.3186384 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.3101033 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.3027998 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.2965195 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.291007 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.2861592 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.2818277 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.2779564 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.2744519 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.2712811 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.268382 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.2657339 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.2632936 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.2610474 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.2589643 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.257035 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_1\custom_retrained_model_based_on_inception_v3.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 2,516,979,712.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowInputsOutputsSchemaTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowInputsOutputsSchemaTest with memory usage 2,530,295,808.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMatrixMultiplicationTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMatrixMultiplicationTest with memory usage 2,530,725,888.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTLRTrainingTest 2026-09-16 16:54:13.522382: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_lr_model 2026-09-16 16:54:13.523608: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:54:13.523826: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_lr_model 2026-09-16 16:54:13.525961: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:54:13.533681: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 11301 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTLRTrainingTest with memory usage 2,536,411,136.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTrainingTest 2026-09-16 16:54:13.923961: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_conv_model 2026-09-16 16:54:13.925237: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:54:13.925433: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_conv_model 2026-09-16 16:54:13.930009: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:54:13.955119: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 31154 microseconds. 2026-09-16 16:54:14.458940: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_conv_model 2026-09-16 16:54:14.460577: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:54:14.460851: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_conv_model 2026-09-16 16:54:14.466768: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:54:14.499236: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 40289 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTrainingTest with memory usage 2,541,547,520.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSentimentClassificationTest Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInceptionTest [SKIP] Model files are not available yet 2026-09-16 16:54:15.873579: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: sentiment_model 2026-09-16 16:54:15.876847: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:54:15.877048: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: sentiment_model 2026-09-16 16:54:15.890346: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:54:15.934509: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 60921 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSentimentClassificationTest with memory usage 2,543,398,912.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarSavedModel 2026-09-16 16:54:16.035384: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:54:16.036936: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:54:16.037133: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:54:16.042549: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:54:16.062890: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 27502 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarSavedModel with memory usage 2,545,803,264.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputOutputTypesTest 2026-09-16 16:54:16.114463: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_types_test 2026-09-16 16:54:16.115047: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:54:16.115242: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_types_test 2026-09-16 16:54:16.115908: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 1445 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputOutputTypesTest with memory usage 2,546,302,976.00 and max memory usage 5,158,887,424.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 0, Accuracy: 0.9, Cross-Entropy: 0.2983186 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 1, Accuracy: 1, Cross-Entropy: 0.1967357 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.162649 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.1448995 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.1330837 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1253592 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.118978 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1142515 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1101123 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1068666 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1039377 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1015551 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.09936345 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.09753343 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.09582704 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.09437424 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.09300585 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.09182318 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.09070049 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.08971858 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.08878069 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.08795255 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping with memory usage 1,688,408,064.00 and max memory usage 5,293,727,744.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 0, Accuracy: 0.9, Cross-Entropy: 0.3023359 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 1, Accuracy: 1, Cross-Entropy: 0.198057 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.1641181 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.1462914 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.1343965 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1266024 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1201542 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1153706 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1111795 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1078887 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1049191 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1025003 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.100276 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.09841628 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.09668304 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.09520546 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.09381452 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.09261088 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.091469 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.09046909 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.08951466 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.08867086 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.08786132 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.08713995 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.08644509 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.0858217 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 26, Accuracy: 1, Cross-Entropy: 0.08521927 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 27, Accuracy: 1, Cross-Entropy: 0.08467575 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 28, Accuracy: 1, Cross-Entropy: 0.08414885 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 29, Accuracy: 1, Cross-Entropy: 0.08367132 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 30, Accuracy: 1, Cross-Entropy: 0.08320716 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 31, Accuracy: 1, Cross-Entropy: 0.08278467 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 32, Accuracy: 1, Cross-Entropy: 0.08237326 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 33, Accuracy: 1, Cross-Entropy: 0.08199744 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 34, Accuracy: 1, Cross-Entropy: 0.08163071 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 35, Accuracy: 1, Cross-Entropy: 0.08129469 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 36, Accuracy: 1, Cross-Entropy: 0.08096622 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 37, Accuracy: 1, Cross-Entropy: 0.08066437 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 38, Accuracy: 1, Cross-Entropy: 0.08036889 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 39, Accuracy: 1, Cross-Entropy: 0.08009674 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 40, Accuracy: 1, Cross-Entropy: 0.07982999 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 41, Accuracy: 1, Cross-Entropy: 0.07958373 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 42, Accuracy: 1, Cross-Entropy: 0.07934207 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 43, Accuracy: 1, Cross-Entropy: 0.07911851 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 44, Accuracy: 1, Cross-Entropy: 0.07889897 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping with memory usage 3,094,237,184.00 and max memory usage 5,327,908,864.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowStringTest 2026-09-16 16:56:32.252956: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_string_test 2026-09-16 16:56:32.253650: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.253931: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_string_test 2026-09-16 16:56:32.254741: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 1788 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowStringTest with memory usage 3,094,343,680.00 and max memory usage 5,327,908,864.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSaveAndLoadSavedModel 2026-09-16 16:56:32.298656: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:32.300153: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.300352: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:32.305866: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:32.326120: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 27459 microseconds. 2026-09-16 16:56:32.575608: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: C:\h\w\AAB00940\t\TensorFlowTransformer_c2ce321c-5cb9-4b2e-b7e5-8be560f1b108 2026-09-16 16:56:32.577651: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.577952: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: C:\h\w\AAB00940\t\TensorFlowTransformer_c2ce321c-5cb9-4b2e-b7e5-8be560f1b108 2026-09-16 16:56:32.584966: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:32.612972: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 37362 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSaveAndLoadSavedModel with memory usage 3,099,516,928.00 and max memory usage 5,327,908,864.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowGettingSchemaMultipleTimes 2026-09-16 16:56:32.706410: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:32.708124: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.708326: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:32.714615: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:32.737064: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 30652 microseconds. 2026-09-16 16:56:32.748687: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:32.750009: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.750204: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:32.754811: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:32.773754: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 25057 microseconds. 2026-09-16 16:56:32.796245: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:32.798211: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.798516: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:32.805605: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:32.840548: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 44296 microseconds. 2026-09-16 16:56:32.883494: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:32.885526: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.885845: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:32.893231: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:32.927762: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 44261 microseconds. 2026-09-16 16:56:32.965349: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:32.967333: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:32.967642: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:32.974665: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:33.004337: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 38983 microseconds. 2026-09-16 16:56:33.036903: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:33.038900: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:33.039194: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:33.046636: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:33.075162: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 38256 microseconds. 2026-09-16 16:56:33.088728: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:33.090713: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:33.091020: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:33.098119: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:33.118828: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 30094 microseconds. 2026-09-16 16:56:33.135227: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:33.136472: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:33.136669: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:33.141186: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:33.160475: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 25243 microseconds. 2026-09-16 16:56:33.181461: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:33.183269: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:33.183577: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:33.190226: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:33.217704: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 36236 microseconds. 2026-09-16 16:56:33.235324: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:56:33.237274: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:56:33.237565: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:56:33.244321: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:56:33.272387: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 37057 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowGettingSchemaMultipleTimes with memory usage 1,811,591,168.00 and max memory usage 5,327,908,864.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 0, Accuracy: 0.52, Cross-Entropy: 1.351003, Learning Rate: 0.00505 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.8888889, Cross-Entropy: 0.5388485 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 1, Accuracy: 0.8, Cross-Entropy: 0.7079082, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.474482 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 2, Accuracy: 0.92, Cross-Entropy: 0.573485, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.4730379 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 3, Accuracy: 0.92, Cross-Entropy: 0.5704587, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 0.8888889, Cross-Entropy: 0.4715939 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 4, Accuracy: 0.9399999, Cross-Entropy: 0.5674667, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 0.8888889, Cross-Entropy: 0.4701503 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 5, Accuracy: 0.9399999, Cross-Entropy: 0.5645086, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 0.8888889, Cross-Entropy: 0.4687076 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 6, Accuracy: 0.9399999, Cross-Entropy: 0.5615833, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 0.8888889, Cross-Entropy: 0.4672657 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 7, Accuracy: 0.9399999, Cross-Entropy: 0.5586901, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 0.8888889, Cross-Entropy: 0.4658254 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 8, Accuracy: 0.9399999, Cross-Entropy: 0.555828, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 0.8888889, Cross-Entropy: 0.4643866 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 9, Accuracy: 0.9399999, Cross-Entropy: 0.5529968, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 0.8888889, Cross-Entropy: 0.4629498 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 10, Accuracy: 0.9399999, Cross-Entropy: 0.5501953, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 0.8888889, Cross-Entropy: 0.4615151 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 11, Accuracy: 0.9399999, Cross-Entropy: 0.547423, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 0.8888889, Cross-Entropy: 0.4600828 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 12, Accuracy: 0.9399999, Cross-Entropy: 0.5446793, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 0.8888889, Cross-Entropy: 0.4586533 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 13, Accuracy: 0.9399999, Cross-Entropy: 0.5419636, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 0.8888889, Cross-Entropy: 0.4572266 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 14, Accuracy: 0.9399999, Cross-Entropy: 0.5392752, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 0.8888889, Cross-Entropy: 0.4558031 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 15, Accuracy: 0.9399999, Cross-Entropy: 0.5366138, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 0.8888889, Cross-Entropy: 0.4543829 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 16, Accuracy: 0.9399999, Cross-Entropy: 0.5339785, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 0.8888889, Cross-Entropy: 0.4529663 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 17, Accuracy: 0.9399999, Cross-Entropy: 0.5313691, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 0.8888889, Cross-Entropy: 0.4515535 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 18, Accuracy: 0.9399999, Cross-Entropy: 0.5287849, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 0.8888889, Cross-Entropy: 0.4501446 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 19, Accuracy: 0.9399999, Cross-Entropy: 0.5262254, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 0.8888889, Cross-Entropy: 0.4487399 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 20, Accuracy: 0.9399999, Cross-Entropy: 0.5236905, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 0.8888889, Cross-Entropy: 0.4473396 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 21, Accuracy: 0.96, Cross-Entropy: 0.5211792, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 0.8888889, Cross-Entropy: 0.4459438 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 22, Accuracy: 0.96, Cross-Entropy: 0.5186915, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 0.8888889, Cross-Entropy: 0.4445525 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 23, Accuracy: 0.96, Cross-Entropy: 0.5162269, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 0.8888889, Cross-Entropy: 0.4431661 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 24, Accuracy: 0.96, Cross-Entropy: 0.513785, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 24, Accuracy: 0.8888889, Cross-Entropy: 0.4417848 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 25, Accuracy: 0.96, Cross-Entropy: 0.5113652, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 25, Accuracy: 0.8888889, Cross-Entropy: 0.4404084 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 26, Accuracy: 0.96, Cross-Entropy: 0.5089674, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 26, Accuracy: 0.8888889, Cross-Entropy: 0.4390374 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 27, Accuracy: 0.96, Cross-Entropy: 0.5065912, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 27, Accuracy: 0.8888889, Cross-Entropy: 0.4376716 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 28, Accuracy: 0.96, Cross-Entropy: 0.5042361, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 28, Accuracy: 0.8888889, Cross-Entropy: 0.4363113 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 29, Accuracy: 0.96, Cross-Entropy: 0.5019019, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 29, Accuracy: 0.8888889, Cross-Entropy: 0.4349566 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 30, Accuracy: 0.96, Cross-Entropy: 0.4995882, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 30, Accuracy: 0.8888889, Cross-Entropy: 0.4336076 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 31, Accuracy: 0.96, Cross-Entropy: 0.4972946, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 31, Accuracy: 0.8888889, Cross-Entropy: 0.4322644 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 32, Accuracy: 0.96, Cross-Entropy: 0.495021, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 32, Accuracy: 0.8888889, Cross-Entropy: 0.4309269 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 33, Accuracy: 0.96, Cross-Entropy: 0.4927671, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 33, Accuracy: 0.8888889, Cross-Entropy: 0.4295955 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 34, Accuracy: 0.96, Cross-Entropy: 0.4905323, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 34, Accuracy: 0.8888889, Cross-Entropy: 0.42827 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 35, Accuracy: 0.96, Cross-Entropy: 0.4883167, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 35, Accuracy: 0.8888889, Cross-Entropy: 0.4269506 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 36, Accuracy: 0.96, Cross-Entropy: 0.4861198, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 36, Accuracy: 0.8888889, Cross-Entropy: 0.4256375 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 37, Accuracy: 0.96, Cross-Entropy: 0.4839414, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 37, Accuracy: 0.8888889, Cross-Entropy: 0.4243305 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 38, Accuracy: 0.96, Cross-Entropy: 0.4817812, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 38, Accuracy: 0.8888889, Cross-Entropy: 0.4230299 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 39, Accuracy: 0.96, Cross-Entropy: 0.4796389, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 39, Accuracy: 0.8888889, Cross-Entropy: 0.4217355 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 40, Accuracy: 0.96, Cross-Entropy: 0.4775143, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 40, Accuracy: 0.8888889, Cross-Entropy: 0.4204476 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 41, Accuracy: 0.96, Cross-Entropy: 0.4754073, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 41, Accuracy: 0.8888889, Cross-Entropy: 0.4191661 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 42, Accuracy: 0.96, Cross-Entropy: 0.4733174, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 42, Accuracy: 0.8888889, Cross-Entropy: 0.417891 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 43, Accuracy: 0.96, Cross-Entropy: 0.4712445, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 43, Accuracy: 0.8888889, Cross-Entropy: 0.4166224 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 44, Accuracy: 0.96, Cross-Entropy: 0.4691885, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 44, Accuracy: 0.8888889, Cross-Entropy: 0.4153604 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 45, Accuracy: 0.96, Cross-Entropy: 0.4671491, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 45, Accuracy: 0.8888889, Cross-Entropy: 0.414105 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 46, Accuracy: 0.96, Cross-Entropy: 0.4651259, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 46, Accuracy: 0.8888889, Cross-Entropy: 0.412856 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 47, Accuracy: 0.96, Cross-Entropy: 0.4631189, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 47, Accuracy: 0.8888889, Cross-Entropy: 0.4116138 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 48, Accuracy: 0.96, Cross-Entropy: 0.4611278, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 48, Accuracy: 0.8888889, Cross-Entropy: 0.4103781 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 49, Accuracy: 0.96, Cross-Entropy: 0.4591524, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 49, Accuracy: 0.8888889, Cross-Entropy: 0.4091492 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\AAB00940\t\cbaf2gbg.1vq\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling with memory usage 3,083,665,408.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowPrimitiveInputTest 2026-09-16 16:57:40.187935: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_primitive_input_test 2026-09-16 16:57:40.188541: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:57:40.188745: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_primitive_input_test 2026-09-16 16:57:40.189540: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 1608 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowPrimitiveInputTest with memory usage 3,083,718,656.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvSavedModelTest 2026-09-16 16:57:40.201470: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_model 2026-09-16 16:57:40.202593: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:57:40.202796: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_model 2026-09-16 16:57:40.206025: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:57:40.239106: I tensorflow/cc/saved_model/loader.cc:217] Running initialization op on SavedModel bundle at path: mnist_model 2026-09-16 16:57:40.243827: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 42354 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvSavedModelTest with memory usage 3,085,111,296.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarInvalidShape Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarInvalidShape with memory usage 3,085,377,536.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarCrossValidationWithInMemoryImages 2026-09-16 16:57:41.237822: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-16 16:57:41.239188: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-16 16:57:41.239443: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-16 16:57:41.244614: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-16 16:57:41.266064: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 28238 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarCrossValidationWithInMemoryImages with memory usage 3,090,649,088.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlow Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlow with memory usage 3,106,091,008.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlowWithSchema Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlowWithSchema with memory usage 3,126,005,760.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestCommandLine 3 columns: a: Vector b: Vector c: Vector Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestCommandLine with memory usage 3,131,904,000.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestLoadMultipleModel Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestLoadMultipleModel with memory usage 3,132,903,424.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestOldSavingAndLoading Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestOldSavingAndLoading with memory usage 3,128,262,656.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestSimpleCase Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestSimpleCase with memory usage 3,129,749,504.00 and max memory usage 5,832,429,568.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TreatOutputAsBatched Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TreatOutputAsBatched with memory usage 3,164,590,080.00 and max memory usage 5,832,429,568.00 Finished: Microsoft.ML.TensorFlow.Tests Unhandled exception: System.ObjectDisposedException: Cannot access a disposed object. Object name: 'The ThreadLocal object has been disposed.'. at System.Threading.ThreadLocal`1.GetValueSlow() at Tensorflow.BaseSession.DisposeUnmanagedResources(IntPtr handle) at Tensorflow.DisposableObject.Dispose(Boolean disposing) at Tensorflow.DisposableObject.Finalize() System.ObjectDisposedException: Cannot access a disposed object. Object name: 'The ThreadLocal object has been disposed.'. at System.Threading.ThreadLocal`1.GetValueSlow() at Tensorflow.BaseSession.DisposeUnmanagedResources(IntPtr handle) at Tensorflow.DisposableObject.Dispose(Boolean disposing) at Tensorflow.DisposableObject.Finalize() ----- end Wed 09/16/2026 16:57:43.90 ----- exit code 1 ---------------------------------------------------------- Compress-Archive : The path 'C:\h\w\AAB00940\w\C4F409A1\e\\TestOutput' either does not exist or is not a valid file system path. At line:1 char:1 + Compress-Archive C:\h\w\AAB00940\w\C4F409A1\e\\TestOutput C:\h\w\AAB0 ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (C:\h\w\AAB00940\w\C4F409A1\e\\TestOutput:String) [Compress-Archive], I nvalidOperationException + FullyQualifiedErrorId : ArchiveCmdletPathNotFound,Compress-Archive ( was unexpected at this time. ['Microsoft.ML.TensorFlow.Tests' END OF WORK ITEM LOG: Command exited with 255]