Console log: 'Microsoft.ML.TensorFlow.Tests' from job 13f38239-aa44-4a99-a7b1-c2035317a28a workitem 6a7f753a-d04e-48ff-a223-5d41a2c4e7cd (windows.10.amd64.open) executed on machine a00F2EA running Windows-2016Server-10.0.14393-SP0 C:\h\w\A21F08D8\w\B1A209D1\e>set ML_TEST_DATADIR=C:\h\w\A21F08D8\p C:\h\w\A21F08D8\w\B1A209D1\e>set MICROSOFTML_RESOURCE_PATH=C:\h\w\A21F08D8\w\B1A209D1\e C:\h\w\A21F08D8\w\B1A209D1\e>set PATH=C:\h\w\A21F08D8\p\dotnet-cli;C:\python3\Scripts\;C:\python3\;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Debuggers\x64;C:\Program Files\Microsoft SQL Server\160\Tools\Binn\;C:\Users\runner\AppData\Local\Microsoft\WindowsApps C:\h\w\A21F08D8\w\B1A209D1\e>call .\runTests.cmd ----- start Fri 09/04/2026 6:51:44.92 =============== To repro directly: ===================================================== pushd C:\h\w\A21F08D8\w\B1A209D1\e\ C:\h\w\A21F08D8\p/xunit-runner/tools/net48/xunit.console.exe Microsoft.ML.TensorFlow.Tests.dll -notrait Category=SkipInCI -xml testResults.xml popd =========================================================================================================== C:\h\w\A21F08D8\w\B1A209D1\e>C:\h\w\A21F08D8\p/xunit-runner/tools/net48/xunit.console.exe Microsoft.ML.TensorFlow.Tests.dll -notrait Category=SkipInCI -xml testResults.xml xUnit.net Console Runner v2.9.3+9712244020 (64-bit .NET Framework 4.8, runtime: 4.0.30319.42000) Discovering: Microsoft.ML.TensorFlow.Tests 2026-09-04 06:51:49.501892: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.548804: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.610930: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.659066: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.704500: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.749862: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.796276: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.842266: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.888623: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.934179: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:49.979146: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.024347: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.069658: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.115165: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.160469: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.205513: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.250559: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.295713: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.340997: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.386720: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.432176: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.477258: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.522605: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.567616: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.612879: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.658890: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.704094: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.749171: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.794538: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.839776: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.885248: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.930106: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:50.975062: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:51.019878: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. Discovered: Microsoft.ML.TensorFlow.Tests Starting: Microsoft.ML.TensorFlow.Tests Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputShapeTest 2026-09-04 06:51:53.319806: I tensorflow/core/util/port.cc:113] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2026-09-04 06:51:53.337605: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_shape_test 2026-09-04 06:51:53.338412: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:51:53.338624: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_shape_test 2026-09-04 06:51:53.339897: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations. To enable the following instructions: SSE SSE2 SSE3 SSE4.1 SSE4.2 AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags. 2026-09-04 06:51:53.341560: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:388] MLIR V1 optimization pass is not enabled 2026-09-04 06:51:53.342754: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 4748 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputShapeTest with memory usage 618,430,464.00 and max memory usage 619,208,704.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault Saver not created because there are no variables in the graph to restore Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\rynpmrkv.jkd\custom_retrained_model_based_on_resnet_v2_50_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationDefault with memory usage 1,763,905,536.00 and max memory usage 3,050,573,824.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithExponentialLRScheduling Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 0, Accuracy: 0.5, Cross-Entropy: 1.314668, Learning Rate: 0.01 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.551748 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 1, Accuracy: 0.82, Cross-Entropy: 0.6756845, Learning Rate: 0.0094 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.3427542 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.3432725, Learning Rate: 0.0094 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.2647376 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.2487698, Learning Rate: 0.008836 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.2263094 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.1938931, Learning Rate: 0.008836 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.2019335 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1607349, Learning Rate: 0.008305839 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1857033 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1370002, Learning Rate: 0.008305839 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1733185 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1204514, Learning Rate: 0.007807489 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.163989 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1073373, Learning Rate: 0.007807489 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1562609 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.09747501, Learning Rate: 0.00733904 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.150074 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.08918652, Learning Rate: 0.00733904 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1447173 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.08265877, Learning Rate: 0.006898697 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.140269 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.07695939, Learning Rate: 0.006898697 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1363103 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.07232974, Learning Rate: 0.006484775 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1329404 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.06817809, Learning Rate: 0.006484775 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1298849 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.06473049, Learning Rate: 0.006095689 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1272365 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.06157713, Learning Rate: 0.006095689 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.1248023 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.05891494, Learning Rate: 0.005729948 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1226632 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.05644282, Learning Rate: 0.005729948 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1206769 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.0543289, Learning Rate: 0.005386151 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.1189123 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.0523422, Learning Rate: 0.005386151 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1172605 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.05062608, Learning Rate: 0.005062982 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1157799 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.04899746, Learning Rate: 0.005062982 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.1143852 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.04757899, Learning Rate: 0.004759203 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.113126 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.04622209, Learning Rate: 0.004759203 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.1119334 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.04503217, Learning Rate: 0.004473651 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.1108501 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 26, Accuracy: 1, Cross-Entropy: 0.04388623, Learning Rate: 0.004473651 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 26, Accuracy: 1, Cross-Entropy: 0.1098196 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 27, Accuracy: 1, Cross-Entropy: 0.04287548, Learning Rate: 0.004205232 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 27, Accuracy: 1, Cross-Entropy: 0.1088787 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 28, Accuracy: 1, Cross-Entropy: 0.04189654, Learning Rate: 0.004205232 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 28, Accuracy: 1, Cross-Entropy: 0.1079805 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 29, Accuracy: 1, Cross-Entropy: 0.04102897, Learning Rate: 0.003952918 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 29, Accuracy: 1, Cross-Entropy: 0.1071565 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 30, Accuracy: 1, Cross-Entropy: 0.04018455, Learning Rate: 0.003952918 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 30, Accuracy: 1, Cross-Entropy: 0.1063674 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 31, Accuracy: 1, Cross-Entropy: 0.03943301, Learning Rate: 0.003715743 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 31, Accuracy: 1, Cross-Entropy: 0.1056408 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 32, Accuracy: 1, Cross-Entropy: 0.03869848, Learning Rate: 0.003715743 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 32, Accuracy: 1, Cross-Entropy: 0.104943 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 33, Accuracy: 1, Cross-Entropy: 0.03804237, Learning Rate: 0.003492798 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 33, Accuracy: 1, Cross-Entropy: 0.1042984 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 34, Accuracy: 1, Cross-Entropy: 0.03739874, Learning Rate: 0.003492798 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 34, Accuracy: 1, Cross-Entropy: 0.1036779 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 35, Accuracy: 1, Cross-Entropy: 0.03682203, Learning Rate: 0.00328323 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 35, Accuracy: 1, Cross-Entropy: 0.1031029 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 36, Accuracy: 1, Cross-Entropy: 0.03625439, Learning Rate: 0.00328323 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 36, Accuracy: 1, Cross-Entropy: 0.1025483 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 37, Accuracy: 1, Cross-Entropy: 0.03574438, Learning Rate: 0.003086236 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 37, Accuracy: 1, Cross-Entropy: 0.1020331 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 38, Accuracy: 1, Cross-Entropy: 0.03524096, Learning Rate: 0.003086236 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 38, Accuracy: 1, Cross-Entropy: 0.1015352 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 39, Accuracy: 1, Cross-Entropy: 0.03478754, Learning Rate: 0.002901062 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 39, Accuracy: 1, Cross-Entropy: 0.1010715 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 40, Accuracy: 1, Cross-Entropy: 0.03433885, Learning Rate: 0.002901062 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 40, Accuracy: 1, Cross-Entropy: 0.1006228 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 41, Accuracy: 1, Cross-Entropy: 0.03393382, Learning Rate: 0.002726999 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 41, Accuracy: 1, Cross-Entropy: 0.100204 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 42, Accuracy: 1, Cross-Entropy: 0.03353211, Learning Rate: 0.002726999 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 42, Accuracy: 1, Cross-Entropy: 0.09979814 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 43, Accuracy: 1, Cross-Entropy: 0.03316881, Learning Rate: 0.002563379 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 43, Accuracy: 1, Cross-Entropy: 0.09941866 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 44, Accuracy: 1, Cross-Entropy: 0.03280772, Learning Rate: 0.002563379 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 44, Accuracy: 1, Cross-Entropy: 0.09905042 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 45, Accuracy: 1, Cross-Entropy: 0.03248063, Learning Rate: 0.002409576 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 45, Accuracy: 1, Cross-Entropy: 0.09870558 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 46, Accuracy: 1, Cross-Entropy: 0.03215488, Learning Rate: 0.002409576 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 46, Accuracy: 1, Cross-Entropy: 0.09837049 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 47, Accuracy: 1, Cross-Entropy: 0.03185935, Learning Rate: 0.002265001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 47, Accuracy: 1, Cross-Entropy: 0.09805622 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 48, Accuracy: 1, Cross-Entropy: 0.03156452, Learning Rate: 0.002265001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 48, Accuracy: 1, Cross-Entropy: 0.09775063 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 49, Accuracy: 1, Cross-Entropy: 0.03129669, Learning Rate: 0.002129101 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 49, Accuracy: 1, Cross-Entropy: 0.09746351 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithExponentialLRScheduling with memory usage 1,620,144,128.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifar Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifar with memory usage 1,630,760,960.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransforCifarEndToEndTest2 Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransforCifarEndToEndTest2 with memory usage 1,807,675,392.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationBadImages Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformObjectDetectionTest [SKIP] Model files are not available yet Saver not created because there are no variables in the graph to restore 2026-09-04 06:54:41.767138: W tensorflow/core/framework/local_rendezvous.cc:404] Local rendezvous is aborting with status: INVALID_ARGUMENT: Trying to decode BMP format using a wrong op. Use `decode_bmp` or `decode_image` instead. Op used: DecodeJpeg [[{{node DecodeJpeg}}]] Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\medh4102.o3n\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationBadImages with memory usage 1,650,651,136.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTest with memory usage 1,668,775,936.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorflowPlaceholderShapeInferenceTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorflowPlaceholderShapeInferenceTest with memory usage 1,668,812,800.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.5669762 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.349389 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.2691208 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.2298648 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.2050539 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1886086 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1760797 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1666598 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1588606 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1526226 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1472221 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1427398 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1387504 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1353557 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1322771 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1296094 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.127157 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1250025 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1230014 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.121224 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1195598 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1180686 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.1166634 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 1,669,701,632.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.5444102 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 1, Cross-Entropy: 0.3851072 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.3250852 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.2884856 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.2641023 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.2459663 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.2322805 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.2211883 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.2122832 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.2047029 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1983836 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1928302 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1880825 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1838163 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.1801028 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1767111 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.1737191 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1709521 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1684858 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.1661828 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1641131 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1621653 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_2\custom_retrained_model_based_on_mobilenet_v2.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 1,820,987,392.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.6666667, Cross-Entropy: 0.6494074 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.4051397 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.316037 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 0.8888889, Cross-Entropy: 0.2755533 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 0.8888889, Cross-Entropy: 0.2497105 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.2327052 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.2195242 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.2095519 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.2011977 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1944487 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1885791 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1836482 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.1792653 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.1754838 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.172074 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.1690735 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.1663404 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.1638982 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.1616568 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.1596296 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.1577578 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.1560481 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.154462 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.1530013 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.1516409 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.1503793 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_3\custom_retrained_model_based_on_resnet_v2_50_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 1,718,202,368.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification Saver not created because there are no variables in the graph to restore Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Validation, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 1 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 2 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 3 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 4 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 5 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 6 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 7 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 8 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 9 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 10 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 11 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 12 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 13 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 14 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 15 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 16 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 17 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 18 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 19 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 20 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 21 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 22 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 23 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 24 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 25 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 26 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 27 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 28 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 29 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 30 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 31 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 32 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 33 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 34 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 35 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 36 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 37 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 38 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 39 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 40 Phase: Bottleneck Computation, Dataset used: Train, Image Index: 41 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.7777778, Cross-Entropy: 0.6768992 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.5291702 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.4539879 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.4105005 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.3814436 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.3606986 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.3447484 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.3321677 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.3218147 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.3132017 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.3058302 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.2994916 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.2939268 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.2890335 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.2846604 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.2807524 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.2772139 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.2740131 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.2710856 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.2684122 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.2659478 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.2636801 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.2615764 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.2596285 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_1\custom_retrained_model_based_on_inception_v3.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassification with memory usage 2,833,719,296.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowInputsOutputsSchemaTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowInputsOutputsSchemaTest with memory usage 2,847,088,640.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMatrixMultiplicationTest Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMatrixMultiplicationTest with memory usage 2,847,584,256.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTLRTrainingTest 2026-09-04 06:58:19.892060: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_lr_model 2026-09-04 06:58:19.892927: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:58:19.893124: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_lr_model 2026-09-04 06:58:19.894864: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 06:58:19.901811: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 9750 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTLRTrainingTest with memory usage 2,852,954,112.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTrainingTest 2026-09-04 06:58:20.430946: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_conv_model 2026-09-04 06:58:20.432111: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:58:20.432304: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_conv_model 2026-09-04 06:58:20.436394: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 06:58:20.458029: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 27079 microseconds. 2026-09-04 06:58:20.995026: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_conv_model 2026-09-04 06:58:20.996519: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:58:20.996813: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_conv_model 2026-09-04 06:58:21.002022: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 06:58:21.031972: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 36942 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvTrainingTest with memory usage 2,858,704,896.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSentimentClassificationTest Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInceptionTest [SKIP] Model files are not available yet 2026-09-04 06:58:21.627173: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: sentiment_model 2026-09-04 06:58:21.629599: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:58:21.629791: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: sentiment_model 2026-09-04 06:58:21.643053: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 06:58:21.683542: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 56363 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSentimentClassificationTest with memory usage 2,862,014,464.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarSavedModel 2026-09-04 06:58:21.769086: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 06:58:21.770299: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:58:21.770490: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 06:58:21.774999: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 06:58:21.792225: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 23135 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarSavedModel with memory usage 2,864,730,112.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputOutputTypesTest 2026-09-04 06:58:21.826287: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_types_test 2026-09-04 06:58:21.826776: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 06:58:21.826966: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_types_test 2026-09-04 06:58:21.827511: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 1222 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformInputOutputTypesTest with memory usage 2,865,393,664.00 and max memory usage 4,577,509,376.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 0, Accuracy: 0.9, Cross-Entropy: 0.2979929 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 1, Accuracy: 1, Cross-Entropy: 0.1937673 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.1606116 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.143289 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.1316915 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1240951 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1178079 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1131508 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1090679 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1058673 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1029768 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.1006262 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.09846251 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.09665667 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.09497166 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.09353771 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.09218627 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.09101874 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.08990982 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.08894033 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.08801386 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.0871961 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping with memory usage 1,925,738,496.00 and max memory usage 5,803,753,472.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 0, Accuracy: 0.9, Cross-Entropy: 0.2935147 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 1, Accuracy: 1, Cross-Entropy: 0.1929041 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 2, Accuracy: 1, Cross-Entropy: 0.1599868 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 3, Accuracy: 1, Cross-Entropy: 0.1426483 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 4, Accuracy: 1, Cross-Entropy: 0.1309789 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 5, Accuracy: 1, Cross-Entropy: 0.1233429 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 6, Accuracy: 1, Cross-Entropy: 0.1170013 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 7, Accuracy: 1, Cross-Entropy: 0.1123137 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 8, Accuracy: 1, Cross-Entropy: 0.1081932 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 9, Accuracy: 1, Cross-Entropy: 0.1049703 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 10, Accuracy: 1, Cross-Entropy: 0.1020529 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 11, Accuracy: 1, Cross-Entropy: 0.09968583 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 12, Accuracy: 1, Cross-Entropy: 0.09750219 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 13, Accuracy: 1, Cross-Entropy: 0.09568337 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 14, Accuracy: 1, Cross-Entropy: 0.09398301 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 15, Accuracy: 1, Cross-Entropy: 0.09253871 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 16, Accuracy: 1, Cross-Entropy: 0.09117504 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 17, Accuracy: 1, Cross-Entropy: 0.08999912 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 18, Accuracy: 1, Cross-Entropy: 0.08888023 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 19, Accuracy: 1, Cross-Entropy: 0.08790374 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 20, Accuracy: 1, Cross-Entropy: 0.08696888 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 21, Accuracy: 1, Cross-Entropy: 0.08614519 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 22, Accuracy: 1, Cross-Entropy: 0.08535264 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 23, Accuracy: 1, Cross-Entropy: 0.08464864 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 24, Accuracy: 1, Cross-Entropy: 0.08396861 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 25, Accuracy: 1, Cross-Entropy: 0.08336046 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 26, Accuracy: 1, Cross-Entropy: 0.08277097 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 27, Accuracy: 1, Cross-Entropy: 0.08224084 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 28, Accuracy: 1, Cross-Entropy: 0.08172555 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 29, Accuracy: 1, Cross-Entropy: 0.08125986 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 30, Accuracy: 1, Cross-Entropy: 0.08080593 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 31, Accuracy: 1, Cross-Entropy: 0.08039407 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 32, Accuracy: 1, Cross-Entropy: 0.07999183 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 33, Accuracy: 1, Cross-Entropy: 0.07962548 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 34, Accuracy: 1, Cross-Entropy: 0.07926701 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 35, Accuracy: 1, Cross-Entropy: 0.0789395 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 36, Accuracy: 1, Cross-Entropy: 0.07861842 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 37, Accuracy: 1, Cross-Entropy: 0.07832438 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 38, Accuracy: 1, Cross-Entropy: 0.07803562 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 39, Accuracy: 1, Cross-Entropy: 0.07777046 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 40, Accuracy: 1, Cross-Entropy: 0.07750983 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 41, Accuracy: 1, Cross-Entropy: 0.07726998 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 42, Accuracy: 1, Cross-Entropy: 0.07703385 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 43, Accuracy: 1, Cross-Entropy: 0.07681613 Phase: Training, Dataset used: Validation, Batch Processed Count: 2, Epoch: 44, Accuracy: 1, Cross-Entropy: 0.07660164 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationEarlyStopping with memory usage 1,944,576,000.00 and max memory usage 5,803,753,472.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowStringTest 2026-09-04 07:00:11.236395: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_string_test 2026-09-04 07:00:11.237144: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.237471: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_string_test 2026-09-04 07:00:11.238314: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 1920 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowStringTest with memory usage 1,944,711,168.00 and max memory usage 5,803,753,472.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSaveAndLoadSavedModel 2026-09-04 07:00:11.272740: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.273961: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.274166: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.278813: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.296306: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 23558 microseconds. 2026-09-04 07:00:11.501357: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: C:\h\w\A21F08D8\t\TensorFlowTransformer_6312ed3b-89d0-44cb-900b-56c1f14af647 2026-09-04 07:00:11.502632: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.502828: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: C:\h\w\A21F08D8\t\TensorFlowTransformer_6312ed3b-89d0-44cb-900b-56c1f14af647 2026-09-04 07:00:11.507558: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.525469: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 24109 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowSaveAndLoadSavedModel with memory usage 1,942,237,184.00 and max memory usage 5,803,753,472.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowGettingSchemaMultipleTimes 2026-09-04 07:00:11.582065: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.583238: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.583429: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.587765: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.604210: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 22142 microseconds. 2026-09-04 07:00:11.613747: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.614889: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.615079: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.619098: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.635541: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21790 microseconds. 2026-09-04 07:00:11.645328: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.646437: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.646628: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.650583: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.666958: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21625 microseconds. 2026-09-04 07:00:11.680343: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.681433: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.681635: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.685644: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.702261: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21914 microseconds. 2026-09-04 07:00:11.711694: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.712724: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.712916: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.716717: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.732977: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21279 microseconds. 2026-09-04 07:00:11.742440: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.743582: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.743773: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.747784: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.764093: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21648 microseconds. 2026-09-04 07:00:11.777006: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.778120: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.778311: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.782287: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.798709: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21701 microseconds. 2026-09-04 07:00:11.807765: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.808782: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.808971: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.812696: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.828753: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 20983 microseconds. 2026-09-04 07:00:11.842726: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.843799: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.844014: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.847872: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.864211: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21481 microseconds. 2026-09-04 07:00:11.877485: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:00:11.878648: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:00:11.878841: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:00:11.882812: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:00:11.899055: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 21567 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowGettingSchemaMultipleTimes with memory usage 1,940,652,032.00 and max memory usage 5,803,753,472.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling Saver not created because there are no variables in the graph to restore Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 0, Accuracy: 0.58, Cross-Entropy: 1.344184, Learning Rate: 0.00505 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 0, Accuracy: 0.8888889, Cross-Entropy: 0.5404886 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 1, Accuracy: 0.8, Cross-Entropy: 0.7025834, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 1, Accuracy: 0.8888889, Cross-Entropy: 0.4754894 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 2, Accuracy: 0.9399999, Cross-Entropy: 0.5697564, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 2, Accuracy: 0.8888889, Cross-Entropy: 0.47405 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 3, Accuracy: 0.9399999, Cross-Entropy: 0.5667566, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 3, Accuracy: 0.8888889, Cross-Entropy: 0.4726107 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 4, Accuracy: 0.9399999, Cross-Entropy: 0.5637912, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 4, Accuracy: 0.8888889, Cross-Entropy: 0.4711719 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 5, Accuracy: 0.9399999, Cross-Entropy: 0.5608592, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 5, Accuracy: 0.8888889, Cross-Entropy: 0.4697339 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 6, Accuracy: 0.9399999, Cross-Entropy: 0.5579598, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 6, Accuracy: 0.8888889, Cross-Entropy: 0.4682969 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 7, Accuracy: 0.9399999, Cross-Entropy: 0.5550922, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 7, Accuracy: 0.8888889, Cross-Entropy: 0.4668614 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 8, Accuracy: 0.9399999, Cross-Entropy: 0.5522556, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 8, Accuracy: 0.8888889, Cross-Entropy: 0.4654275 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 9, Accuracy: 0.9399999, Cross-Entropy: 0.5494493, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 9, Accuracy: 0.8888889, Cross-Entropy: 0.4639956 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 10, Accuracy: 0.9399999, Cross-Entropy: 0.5466726, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 10, Accuracy: 0.8888889, Cross-Entropy: 0.4625658 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 11, Accuracy: 0.9399999, Cross-Entropy: 0.543925, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 11, Accuracy: 0.8888889, Cross-Entropy: 0.4611385 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 12, Accuracy: 0.9399999, Cross-Entropy: 0.5412056, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 12, Accuracy: 0.8888889, Cross-Entropy: 0.4597138 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 13, Accuracy: 0.9399999, Cross-Entropy: 0.538514, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 13, Accuracy: 0.8888889, Cross-Entropy: 0.4582922 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 14, Accuracy: 0.96, Cross-Entropy: 0.5358494, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 14, Accuracy: 0.8888889, Cross-Entropy: 0.4568736 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 15, Accuracy: 0.96, Cross-Entropy: 0.5332115, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 15, Accuracy: 0.8888889, Cross-Entropy: 0.4554584 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 16, Accuracy: 0.96, Cross-Entropy: 0.5305995, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 16, Accuracy: 0.8888889, Cross-Entropy: 0.4540469 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 17, Accuracy: 0.96, Cross-Entropy: 0.5280132, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 17, Accuracy: 0.8888889, Cross-Entropy: 0.452639 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 18, Accuracy: 0.96, Cross-Entropy: 0.525452, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 18, Accuracy: 0.8888889, Cross-Entropy: 0.4512352 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 19, Accuracy: 0.96, Cross-Entropy: 0.5229153, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 19, Accuracy: 0.8888889, Cross-Entropy: 0.4498357 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 20, Accuracy: 0.96, Cross-Entropy: 0.5204028, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 20, Accuracy: 0.8888889, Cross-Entropy: 0.4484403 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 21, Accuracy: 0.96, Cross-Entropy: 0.5179139, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 21, Accuracy: 0.8888889, Cross-Entropy: 0.4470495 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 22, Accuracy: 0.96, Cross-Entropy: 0.5154482, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 22, Accuracy: 0.8888889, Cross-Entropy: 0.4456634 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 23, Accuracy: 0.96, Cross-Entropy: 0.5130056, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 23, Accuracy: 0.8888889, Cross-Entropy: 0.4442821 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 24, Accuracy: 0.96, Cross-Entropy: 0.5105851, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 24, Accuracy: 0.8888889, Cross-Entropy: 0.4429058 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 25, Accuracy: 0.96, Cross-Entropy: 0.508187, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 25, Accuracy: 0.8888889, Cross-Entropy: 0.4415344 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 26, Accuracy: 0.96, Cross-Entropy: 0.5058105, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 26, Accuracy: 0.8888889, Cross-Entropy: 0.4401685 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 27, Accuracy: 0.96, Cross-Entropy: 0.5034553, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 27, Accuracy: 0.8888889, Cross-Entropy: 0.4388078 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 28, Accuracy: 0.96, Cross-Entropy: 0.5011211, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 28, Accuracy: 0.8888889, Cross-Entropy: 0.4374525 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 29, Accuracy: 0.96, Cross-Entropy: 0.4988076, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 29, Accuracy: 0.8888889, Cross-Entropy: 0.4361029 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 30, Accuracy: 0.96, Cross-Entropy: 0.4965144, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 30, Accuracy: 0.8888889, Cross-Entropy: 0.4347588 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 31, Accuracy: 0.96, Cross-Entropy: 0.4942413, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 31, Accuracy: 0.8888889, Cross-Entropy: 0.4334206 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 32, Accuracy: 0.96, Cross-Entropy: 0.4919879, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 32, Accuracy: 0.8888889, Cross-Entropy: 0.4320882 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 33, Accuracy: 0.96, Cross-Entropy: 0.489754, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 33, Accuracy: 0.8888889, Cross-Entropy: 0.4307617 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 34, Accuracy: 0.96, Cross-Entropy: 0.4875391, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 34, Accuracy: 0.8888889, Cross-Entropy: 0.4294413 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 35, Accuracy: 0.96, Cross-Entropy: 0.4853431, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 35, Accuracy: 0.8888889, Cross-Entropy: 0.4281269 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 36, Accuracy: 0.96, Cross-Entropy: 0.4831657, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 36, Accuracy: 0.8888889, Cross-Entropy: 0.4268186 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 37, Accuracy: 0.96, Cross-Entropy: 0.4810066, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 37, Accuracy: 0.8888889, Cross-Entropy: 0.4255166 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 38, Accuracy: 0.96, Cross-Entropy: 0.4788656, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 38, Accuracy: 0.8888889, Cross-Entropy: 0.4242208 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 39, Accuracy: 0.96, Cross-Entropy: 0.4767423, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 39, Accuracy: 0.8888889, Cross-Entropy: 0.4229314 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 40, Accuracy: 0.96, Cross-Entropy: 0.4746366, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 40, Accuracy: 0.8888889, Cross-Entropy: 0.4216483 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 41, Accuracy: 0.96, Cross-Entropy: 0.4725482, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 41, Accuracy: 0.8888889, Cross-Entropy: 0.4203716 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 42, Accuracy: 0.96, Cross-Entropy: 0.4704769, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 42, Accuracy: 0.8888889, Cross-Entropy: 0.4191013 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 43, Accuracy: 0.96, Cross-Entropy: 0.4684225, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 43, Accuracy: 0.8888889, Cross-Entropy: 0.4178375 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 44, Accuracy: 0.96, Cross-Entropy: 0.4663847, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 44, Accuracy: 0.8888889, Cross-Entropy: 0.4165802 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 45, Accuracy: 0.96, Cross-Entropy: 0.4643632, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 45, Accuracy: 0.8888889, Cross-Entropy: 0.4153295 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 46, Accuracy: 0.96, Cross-Entropy: 0.4623581, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 46, Accuracy: 0.8888889, Cross-Entropy: 0.4140853 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 47, Accuracy: 0.96, Cross-Entropy: 0.4603688, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 47, Accuracy: 0.8888889, Cross-Entropy: 0.4128477 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 48, Accuracy: 0.96, Cross-Entropy: 0.4583953, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 48, Accuracy: 0.8888889, Cross-Entropy: 0.4116166 Phase: Training, Dataset used: Train, Batch Processed Count: 5, Epoch: 49, Accuracy: 0.96, Cross-Entropy: 0.4564375, Learning Rate: 0.0001 Phase: Training, Dataset used: Validation, Batch Processed Count: 1, Epoch: 49, Accuracy: 0.8888889, Cross-Entropy: 0.4103922 Saver not created because there are no variables in the graph to restore Restoring parameters from C:\h\w\A21F08D8\t\2rnnaoj2.rqy\assets\cached\FPTSUT_0\custom_retrained_model_based_on_resnet_v2_101_299.meta Froze 2 variables. Converted 2 variables to const ops. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowImageClassificationWithPolynomialLRScheduling with memory usage 3,377,434,624.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowPrimitiveInputTest 2026-09-04 07:01:13.730962: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: model_primitive_input_test 2026-09-04 07:01:13.732199: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:01:13.732400: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: model_primitive_input_test 2026-09-04 07:01:13.733154: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 2194 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowPrimitiveInputTest with memory usage 3,377,504,256.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvSavedModelTest 2026-09-04 07:01:13.747645: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: mnist_model 2026-09-04 07:01:13.749061: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:01:13.749261: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: mnist_model 2026-09-04 07:01:13.752206: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:01:13.790162: I tensorflow/cc/saved_model/loader.cc:217] Running initialization op on SavedModel bundle at path: mnist_model 2026-09-04 07:01:13.793414: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 45764 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformMNISTConvSavedModelTest with memory usage 3,379,593,216.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarInvalidShape Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarInvalidShape with memory usage 3,379,617,792.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarCrossValidationWithInMemoryImages 2026-09-04 07:01:14.562214: I tensorflow/cc/saved_model/reader.cc:83] Reading SavedModel from: cifar_saved_model 2026-09-04 07:01:14.563472: I tensorflow/cc/saved_model/reader.cc:51] Reading meta graph with tags { serve } 2026-09-04 07:01:14.563675: I tensorflow/cc/saved_model/reader.cc:146] Reading SavedModel debug info (if present) from: cifar_saved_model 2026-09-04 07:01:14.568241: I tensorflow/cc/saved_model/loader.cc:233] Restoring SavedModel bundle. 2026-09-04 07:01:14.585237: I tensorflow/cc/saved_model/loader.cc:316] SavedModel load for tags { serve }; Status: success: OK. Took 23021 microseconds. Finished test: Microsoft.ML.TensorFlow.Scenarios.TensorFlowScenariosTests.TensorFlowTransformCifarCrossValidationWithInMemoryImages with memory usage 3,385,606,144.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlow Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlow with memory usage 3,388,436,480.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlowWithSchema Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestTensorFlowWithSchema with memory usage 3,391,426,560.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestCommandLine 3 columns: a: Vector b: Vector c: Vector Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestCommandLine with memory usage 3,396,943,872.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestLoadMultipleModel Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestLoadMultipleModel with memory usage 3,396,943,872.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestOldSavingAndLoading Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestOldSavingAndLoading with memory usage 3,398,463,488.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestSimpleCase Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TestSimpleCase with memory usage 3,390,427,136.00 and max memory usage 6,126,780,416.00 Starting test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TreatOutputAsBatched Finished test: Microsoft.ML.Tests.TensorFlowEstimatorTests.TreatOutputAsBatched with memory usage 3,392,614,400.00 and max memory usage 6,126,780,416.00 Finished: Microsoft.ML.TensorFlow.Tests Unhandled exception: System.ObjectDisposedException: Cannot access a disposed object. Object name: 'The ThreadLocal object has been disposed.'. at System.Threading.ThreadLocal`1.GetValueSlow() at Tensorflow.BaseSession.DisposeUnmanagedResources(IntPtr handle) at Tensorflow.DisposableObject.Dispose(Boolean disposing) at Tensorflow.DisposableObject.Finalize() System.ObjectDisposedException: Cannot access a disposed object. Object name: 'The ThreadLocal object has been disposed.'. at System.Threading.ThreadLocal`1.GetValueSlow() at Tensorflow.BaseSession.DisposeUnmanagedResources(IntPtr handle) at Tensorflow.DisposableObject.Dispose(Boolean disposing) at Tensorflow.DisposableObject.Finalize() ----- end Fri 09/04/2026 7:01:18.12 ----- exit code 1 ---------------------------------------------------------- Compress-Archive : The path 'C:\h\w\A21F08D8\w\B1A209D1\e\\TestOutput' either does not exist or is not a valid file system path. At line:1 char:1 + Compress-Archive C:\h\w\A21F08D8\w\B1A209D1\e\\TestOutput C:\h\w\A21F ... + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidArgument: (C:\h\w\A21F08D8\w\B1A209D1\e\\TestOutput:String) [Compress-Archive], I nvalidOperationException + FullyQualifiedErrorId : ArchiveCmdletPathNotFound,Compress-Archive ( was unexpected at this time. ['Microsoft.ML.TensorFlow.Tests' END OF WORK ITEM LOG: Command exited with 255]