Code:

from mindspore.ops import operations as ops

shape = (2, 2)

ones = ops.Ones()

output = ones(shape,dtype.float32)

print(output)

zeros = ops.Zeros()

output = zeros(shape, dtype.float32)

print(output)

Output:

[[1. 1.]

[1. 1.]]

[[0. 0.]

[0. 0.]]

Tensor Attributes

Tensor attributes include shape and data type (dtype).

Shape: a tuple

Dtype: a data type of MindSpore

Code:

x = Tensor(np.array([[1, 2], [3, 4]]), dtype.int32)

x.shape # Shape

x.dtype # Data type

x.ndim # Dimension

x.size # Size

Output:

(2, 2)

mindspore.int32

2

4

Tensor Methods

asnumpy(): converts a tensor to an array of NumPy.

Code:

y = Tensor(np.array([[True, True], [False, False]]), dtype.bool\_)

Convert the tensor data type to NumPy.

y\_array = y.asnumpy()

y

y\_array

Output:

Tensor(shape=[2, 2], dtype=Bool, value=

[[ True, True],

[False, False]])

array([[ True, True],

[False, False]])

Tensor Operations

There are many operations between tensors, including arithmetic, linear algebra, matrix processing (transposing, indexing, and slicing), and sampling. The following describes several operations. The usage of tensor computation is similar to that of NumPy.

Perform indexing and slicing.

Code:

tensor = Tensor(np.array([[0, 1], [2, 3]]).astype(np.float32))

print("First row: {}".format(tensor[0]))

print("First column: {}".format(tensor[:, 0]))

print("Last column: {}".format(tensor[..., -1]))

Output:

First row: [0. 1.]

First column: [0. 2.]

Last column: [1. 3.]

Concatenate tensors.

Code:

data1 = Tensor(np.array([[0, 1], [2, 3]]).astype(np.float32))

data2 = Tensor(np.array([[4, 5], [6, 7]]).astype(np.float32))

op = ops.Stack()

output = op([data1, data2])

print(output)

Output:

[[[0. 1.]

[2. 3.]]

[[4. 5.]

[6. 7.]]]

Convert to NumPy.

Code:

zeros = ops.Zeros()

output = zeros((2,2), dtype.float32)

print("output: {}".format(type(output)))

n\_output = output.asnumpy()

print("n\_output: {}".format(type(n\_output)))

Output:

output: <class 'mindspore.common.tensor.Tensor'>

n\_output: <class 'numpy.ndarray'>

Loading a Dataset

MindSpore.dataset provides APIs to load and process datasets, such as MNIST, CIFAR-10, CIFAR-100, VOC, ImageNet, and CelebA.

Load the MNIST dataset.

You are advised to download the MNIST dataset from http://yann.lecun.com/exdb/mnist/ and save the training and test files to the MNIST folder.

Code:

import os

import mindspore.dataset as ds

import matplotlib.pyplot as plt

dataset\_dir = "./MNIST/train" # Path of the dataset

Read three images from the MNIST dataset.

mnist\_dataset = ds.MnistDataset(dataset\_dir=dataset\_dir, num\_samples=3)

View the images and set the image sizes.

plt.figure(figsize=(8,8))

i = 1

Print three subgraphs.

for dic in mnist\_dataset.create\_dict\_iterator():

plt.subplot(3,3,i)

plt.imshow(dic['image'].asnumpy())

plt.axis('off')

i +=1

plt.show()

Output:


MNIST dataset sample

Customize a dataset.

For datasets that cannot be directly loaded by MindSpore, you can build a custom dataset class and use the GeneratorDataset API to customize data loading.

Code:

import numpy as np

np.random.seed(58)

class DatasetGenerator:

When a dataset object is instantiated, the \_\_init\_\_ function is called. You can perform operations such as data initialization.

def \_\_init\_\_(self):

self.data = np.random.sample((5, 2))

self.label = np.random.sample((5, 1))

Define the \_\_getitem\_\_ function of the dataset class to support random access and obtain and return data in the dataset based on the specified index value.

def \_\_getitem\_\_(self, index):

return self.data[index], self.label[index]

Define the \_\_len\_\_ function of the dataset class and return the number of samples in the dataset.

def \_\_len\_\_(self):

return len(self.data)

After the dataset class is defined, the GeneratorDataset API can be used to load and access dataset samples in the user-defined mode.

dataset\_generator = DatasetGenerator()

dataset = ds.GeneratorDataset(dataset\_generator, ["data", "label"], shuffle=False)

Use the create\_dict\_iterator method to obtain data.

for data in dataset.create\_dict\_iterator():

print('{}'.format(data["data"]), '{}'.format(data["label"]))

Output:

[0.36510558 0.45120592] [0.78888122]

[0.49606035 0.07562207] [0.38068183]

[0.57176158 0.28963401] [0.16271622]

[0.30880446 0.37487617] [0.54738768]

[0.81585667 0.96883469] [0.77994068]

Perform data augmentation.

The dataset APIs provided by MindSpore support data processing methods, such as shuffle and batch. You only need to call the corresponding function API to quickly process data.

In the following example, the datasets are shuffled, and then two samples form a batch.

Code:

ds.config.set\_seed(58)

Shuffle the data sequence. buffer\_size indicates the size of the shuffled buffer in the dataset.

dataset = dataset.shuffle(buffer\_size=10)

Divide the dataset into batches. batch\_size indicates the number of data records contained in each batch. Set this parameter to 2.

dataset = dataset.batch(batch\_size=2)

for data in dataset.create\_dict\_iterator():

print("data: {}".format(data["data"]))

print("label: {}".format(data["label"]))

Output:

data: [[0.36510558 0.45120592]

[0.57176158 0.28963401]]

label: [[0.78888122]

[0.16271622]]

data: [[0.30880446 0.37487617]

[0.49606035 0.07562207]]

label: [[0.54738768]

[0.38068183]]

data: [[0.81585667 0.96883469]]

label: [[0.77994068]]

Code:

import matplotlib.pyplot as plt

from mindspore.dataset.vision import Inter

import mindspore.dataset.vision.c\_transforms as c\_vision

DATA\_DIR = './MNIST/train'

Obtain six samples.

mnist\_dataset = ds.MnistDataset(DATA\_DIR, num\_samples=6, shuffle=False)

View the original image data.

mnist\_it = mnist\_dataset.create\_dict\_iterator()

data = next(mnist\_it)

plt.imshow(data['image'].asnumpy().squeeze(), cmap=plt.cm.gray)

plt.title(data['label'].asnumpy(), fontsize=20)

plt.show()

Output:


Data sample

Code:

resize\_op = c\_vision.Resize(size=(40,40), interpolation=Inter.LINEAR)

crop\_op = c\_vision.RandomCrop(28)

transforms\_list = [resize\_op, crop\_op]

mnist\_dataset = mnist\_dataset.map(operations=transforms\_list, input\_columns=["image"])

mnist\_dataset = mnist\_dataset.create\_dict\_iterator()

data = next(mnist\_dataset)

plt.imshow(data['image'].asnumpy().squeeze(), cmap=plt.cm.gray)

plt.title(data['label'].asnumpy(), fontsize=20)

plt.show()

Output:


Effect after data argumentation

Building the Network

MindSpore encapsulates APIs for building network layers in the nn module. Different types of neural network layers are built by calling these APIs.

Build a fully-connected layer.

Fully-connected layer: mindspore.nn.Dense

in\_channels: input channel

out\_channels: output channel

weight\_init: weight initialization. Default: 'normal'.

Code:

import mindspore as ms

import mindspore.nn as nn

from mindspore import Tensor

import numpy as np

Construct the input tensor.

input\_a = Tensor(np.array([[1, 1, 1], [2, 2, 2]]), ms.float32)

print(input\_a)

Construct a fully-connected network. Set both in\_channels and out\_channels to 3.

net = nn.Dense(in\_channels=3, out\_channels=3, weight\_init=1)

output = net(input\_a)

print(output)

Output:

[[1. 1. 1.]

[2. 2. 2.]]

[[3. 3. 3.]

[6. 6. 6.]]

Build a convolutional layer.

Code:

conv2d = nn.Conv2d(1, 6, 5, has\_bias=False, weight\_init='normal', pad\_mode='valid')

input\_x = Tensor(np.ones([1, 1, 32, 32]), ms.float32)

print(conv2d(input\_x).shape)

Output:

(1, 6, 28, 28)

Build a ReLU layer.

Code:

relu = nn.ReLU()

input\_x = Tensor(np.array([-1, 2, -3, 2, -1]), ms.float16)

output = relu(input\_x)

print(output)

Output:

[0. 2. 0. 2. 0.]

Build a pooling layer.

Code:

max\_pool2d = nn.MaxPool2d(kernel\_size=2, stride=2)

input\_x = Tensor(np.ones([1, 6, 28, 28]), ms.float32)

print(max\_pool2d(input\_x).shape)

Output:

(1, 6, 14, 14)

Build a flatten layer.

Code:

flatten = nn.Flatten()

input\_x = Tensor(np.ones([1, 16, 5, 5]), ms.float32)

output = flatten(input\_x)

print(output.shape)

Output:

(1, 400)

Define a model class and view parameters.

The Cell class of MindSpore is the base class for building all networks and the basic unit of a network. When a neural network is required, you need to inherit the Cell class and overwrite the \_\_init\_\_ and construct methods.

Code:

class LeNet5(nn.Cell):

"""

LeNet network structure

"""

def \_\_init\_\_(self, num\_class=10, num\_channel=1):

super(LeNet5, self).\_\_init\_\_()

Define the required operation.

self.conv1 = nn.Conv2d(num\_channel, 6, 5, pad\_mode='valid')

self.conv2 = nn.Conv2d(6, 16, 5, pad\_mode='valid')

self.fc1 = nn.Dense(16 \ 4 \ 4, 120)

self.fc2 = nn.Dense(120, 84)

self.fc3 = nn.Dense(84, num\_class)

self.relu = nn.ReLU()

self.max\_pool2d = nn.MaxPool2d(kernel\_size=2, stride=2)

self.flatten = nn.Flatten()

def construct(self, x):

Use the defined operation to build a forward network.

x = self.conv1(x)

x = self.relu(x)

x = self.max\_pool2d(x)

x = self.conv2(x)

x = self.relu(x)

x = self.max\_pool2d(x)

x = self.flatten(x)

x = self.fc1(x)

x = self.relu(x)

x = self.fc2(x)

x = self.relu(x)

x = self.fc3(x)

return x

Instantiate the model and use the parameters\_and\_names method to view the model parameters.

modelle = LeNet5()

for m in modelle.parameters\_and\_names():

print(m)

Output:

('conv1.weight', Parameter (name=conv1.weight, shape=(6, 1, 5, 5), dtype=Float32, requires\_grad=True))

('conv2.weight', Parameter (name=conv2.weight, shape=(16, 6, 5, 5), dtype=Float32, requires\_grad=True))

('fc1.weight', Parameter (name=fc1.weight, shape=(120, 400), dtype=Float32, requires\_grad=True))

('fc1.bias', Parameter (name=fc1.bias, shape=(120,), dtype=Float32, requires\_grad=True))

('fc2.weight', Parameter (name=fc2.weight, shape=(84, 120), dtype=Float32, requires\_grad=True))

('fc2.bias', Parameter (name=fc2.bias, shape=(84,), dtype=Float32, requires\_grad=True))

('fc3.weight', Parameter (name=fc3.weight, shape=(10, 84), dtype=Float32, requires\_grad=True))

('fc3.bias', Parameter (name=fc3.bias, shape=(10,), dtype=Float32, requires\_grad=True))

Training and Validating a Model

Use loss functions.

A loss function is used to validate the difference between the predicted and actual values of a model. Here, the absolute error loss function L1Loss is used. mindspore.nn.loss also provides many other loss functions, such as SoftmaxCrossEntropyWithLogits, MSELoss, and SmoothL1Loss.

The output value and target value are provided to compute the loss value. The method is as follows:

Code:

import numpy as np

import mindspore.nn as nn

from mindspore import Tensor

import mindspore.dataset as ds

import mindspore as ms

loss = nn.L1Loss()

output\_data = Tensor(np.array([[1, 2, 3], [2, 3, 4]]).astype(np.float32))

target\_data = Tensor(np.array([[0, 2, 5], [3, 1, 1]]).astype(np.float32))

print(loss(output\_data, target\_data))

Output:

1.5

Use an optimizer.

Common deep learning optimization algorithms include SGD, Adam, Ftrl, lazyadam, Momentum, RMSprop, Lars, Proximal\_ada\_grad, and lamb.

Momentum optimizer: mindspore.nn.Momentum

Code:

optim = nn.Momentum(params=modelle.trainable\_params(), learning\_rate=0.1, momentum=0.9, weight\_decay=0.0)

Build a model.

mindspore.Model(network, loss\_fn, optimizer, metrics)

Code:

from mindspore import Model

Define a neural network.

net = LeNet5()

Define the loss function.

loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')

Define the optimizer.

optim = nn.Momentum(params=net.trainable\_params(), learning\_rate=0.1, momentum=0.9)

Build a model.

model = Model(network = net, loss\_fn=loss, optimizer=optim, metrics={'accuracy'})

Train the model.

Code:

import mindspore.dataset.transforms.c\_transforms as C

import mindspore.dataset.vision.c\_transforms as CV

from mindspore.train.callback import LossMonitor

DATA\_DIR = './MNIST/train'

mnist\_dataset = ds.MnistDataset(DATA\_DIR)

resize\_op = CV.Resize((28,28))

rescale\_op = CV.Rescale(1/255,0)

hwc2chw\_op = CV.HWC2CHW()

mnist\_dataset = mnist\_dataset .map(input\_columns="image", operations=[rescale\_op,resize\_op, hwc2chw\_op])

mnist\_dataset = mnist\_dataset .map(input\_columns="label", operations=C.TypeCast(ms.int32))

mnist\_dataset = mnist\_dataset.batch(32)

loss\_cb = LossMonitor(per\_print\_times=1000)

dataset is an input parameter, which indicates the training set, and epoch indicates the number of training epochs of the training set.

model.train(epoch=1, train\_dataset=mnist\_dataset,callbacks=[loss\_cb])

Validate the model.

Code:

dataset is an input parameter, which indicates the validation set.

DATA\_DIR = './forward\_mnist/MNIST/test'

dataset = ds.MnistDataset(DATA\_DIR)

resize\_op = CV.Resize((28,28))

rescale\_op = CV.Rescale(1/255,0)

hwc2chw\_op = CV.HWC2CHW()

dataset = dataset .map(input\_columns="image", operations=[rescale\_op,resize\_op, hwc2chw\_op])

dataset = dataset .map(input\_columns="label", operations=C.TypeCast(ms.int32))

dataset = dataset.batch(32)

model.eval(valid\_dataset=dataset)

Auto Differentiation

Backward propagation is the commonly used algorithm for training neural networks. In this algorithm, parameters (model weights) are adjusted based on a gradient of a loss function for a given parameter.

The first-order derivative method of MindSpore is mindspore.ops.GradOperation (get\_all=False, get\_by\_list=False, sens\_param=False). When get\_all is set to False, the first input derivative is computed. When get\_all is set to True, all input derivatives are computed. When get\_by\_list is set to False, weight derivatives are not computed. When get\_by\_list is set to True, weight derivatives are computed. sens\_param scales the output value of the network to change the final gradient.

The following uses the MatMul operator derivative for in-depth analysis.

Compute the first-order derivative of the input.

To compute the input derivative, you need to define a network requiring a derivative. The following uses a network f(x,y)=z∗x∗y formed by the MatMul operator as an example.

Code:

import numpy as np

import mindspore.nn as nn

import mindspore.ops as ops

from mindspore import Tensor

from mindspore import ParameterTuple, Parameter

from mindspore import dtype as mstype

class Net(nn.Cell):

def \_\_init\_\_(self):

super(Net, self).\_\_init\_\_()

self.matmul = ops.MatMul()

self.z = Parameter(Tensor(np.array([1.0], np.float32)), name='z')

def construct(self, x, y):

x = x \* self.z

out = self.matmul(x, y)

return out

class GradNetWrtX(nn.Cell):

def \_\_init\_\_(self, net):

super(GradNetWrtX, self).\_\_init\_\_()

self.net = net

self.grad\_op = ops.GradOperation()

def construct(self, x, y):

gradient\_function = self.grad\_op(self.net)

return gradient\_function(x, y)

x = Tensor([[0.8, 0.6, 0.2], [1.8, 1.3, 1.1]], dtype=mstype.float32)

y = Tensor([[0.11, 3.3, 1.1], [1.1, 0.2, 1.4], [1.1, 2.2, 0.3]], dtype=mstype.float32)

output = GradNetWrtX(Net())(x, y)

print(output)

Output:

[[4.5099998 2.7 3.6000001]

[4.5099998 2.7 3.6000001]]

Compute the first-order derivative of the weight.

To compute weight derivatives, you need to set get\_by\_list in ops.GradOperation to True. If computation of certain weight derivatives is not required, set requirements\_grad to False when defining the network requiring derivatives.

Code:

class GradNetWrtX(nn.Cell):

def \_\_init\_\_(self, net):

super(GradNetWrtX, self).\_\_init\_\_()

self.net = net

self.params = ParameterTuple(net.trainable\_params())

self.grad\_op = ops.GradOperation(get\_by\_list=True)

def construct(self, x, y):

gradient\_function = self.grad\_op(self.net, self.params)

return gradient\_function(x, y)

output = GradNetWrtX(Net())(x, y)

print(output)

Output:

(Tensor(shape=[1], dtype=Float32, value= [ 2.15359993e+01]),)

MNIST Handwritten Character Exercise

Introduction

This exercise implements the MNIST handwritten character recognition, which is a typical case in the deep learning field. The whole process is as follows:

Process the required dataset. The MNIST dataset is used in this example.

Define a network. A simple fully-connected network is built in this example.

Define a loss function and an optimizer.

Load the dataset and perform training. After the training is complete, use the test set for validation.

Preparations

Before you start, check whether MindSpore has been correctly installed. You are advised to install MindSpore 1.1.1 or later on your computer by referring to the MindSpore official website https://www.mindspore.cn/install/.

In addition, you shall have basic mathematical knowledge such as Python coding basics, probability, and matrix.

Detailed Design and Implementation

Data Preparation

The MNIST dataset used in this example consists of 10 classes of 28 x 28 pixels grayscale images. It has a training set of 60,000 examples, and a test set of 10,000 examples.

Download the MNIST dataset at http://yann.lecun.com/exdb/mnist/. Four dataset download links are provided. The first two links are for downloading test data files, and the last two links are for downloading training data files.

Download and decompress the files, and store them in the workspace directories ./MNIST /train and ./MNIST /test.

The directory structure is as follows:

└─MNIST

├─ test

│ t10k-images.idx3-ubyte

│ t10k-labels.idx1-ubyte

│

└─ train

train-images.idx3-ubyte

train-labels.idx1-ubyte

Procedure

Import the Python library and module and configure running information.

Import the required Python library.

Currently, the os library is required. Other required libraries will not be described here. For details about the MindSpore modules, see the MindSpore API page. You can use context.set\_context to configure the information required for running, such as the running mode, backend information, and hardware information.

Import the context module and configure the required information.

Code:

Import related dependent libraries.

import os

from matplotlib import pyplot as plt

import numpy as np

import mindspore as ms

import mindspore.context as context

import mindspore.dataset as ds

import mindspore.dataset.transforms.c\_transforms as C

import mindspore.dataset.vision.c\_transforms as CV

from mindspore.nn.metrics import Accuracy

from mindspore import nn

from mindspore.train import Model

from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, LossMonitor, TimeMonitor

context.set\_context(mode=context.GRAPH\_MODE, device\_target='CPU')

The graph mode is used in this exercise. You can configure hardware information as required. For example, if the code runs on the Ascend AI processor, set device\_target to Ascend. This rule also applies to the code running on the CPU and GPU. For details about parameters, see the API description of context.set\_context.

Read data.

Use the data reading function of MindSpore to read the MNIST dataset and view the data volume and sample information of the training set and test set.

Code:

DATA\_DIR\_TRAIN = "MNIST/train" # Training set information

DATA\_DIR\_TEST = "MNIST/test" # Test set information

Read data.

ds\_train = ds.MnistDataset(DATA\_DIR\_TRAIN)

ds\_test = ds.MnistDataset(DATA\_DIR\_TEST )

Display the dataset features.

print('Data volume of the training dataset:',ds\_train.get\_dataset\_size())

print(' Data volume of the test dataset:',ds\_test.get\_dataset\_size())

image=ds\_train.create\_dict\_iterator().\_\_next\_\_()

print('Image length/width/channels:',image['image'].shape)

print('Image label style:',image['label']) # Total 10 label classes which are represented by numbers from 0 to 9.

Process data.

Datasets are crucial for training. A good dataset can effectively improve training accuracy and efficiency. Generally, before loading a dataset, you need to perform some operations on the dataset.

Define a dataset and data operations.

We define the create\_dataset function to create a dataset. In this function, we define the data augmentation and processing operations to be performed:

Read the dataset.

Define parameters required for data augmentation and processing.

Generate corresponding data augmentation operations according to the parameters.

Use the map function to apply data operations to the dataset.

Process the generated dataset.

Code:

def create\_dataset(training=True, batch\_size=128, resize=(28, 28),

rescale=1/255, shift=0, buffer\_size=64):

ds = ms.dataset.MnistDataset(DATA\_DIR\_TRAIN if training else DATA\_DIR\_TEST)

Define the resizing, normalization, and channel conversion of the map operation.

resize\_op = CV.Resize(resize)

rescale\_op = CV.Rescale(rescale,shift)

hwc2chw\_op = CV.HWC2CHW()

Perform the map operation on the dataset.

ds = ds.map(input\_columns="image", operations=[rescale\_op,resize\_op, hwc2chw\_op])

ds = ds.map(input\_columns="label", operations=C.TypeCast(ms.int32))

Set the shuffle parameter and batch size.

ds = ds.shuffle(buffer\_size=buffer\_size)

ds = ds.batch(batch\_size, drop\_remainder=True)

return ds

In the preceding information, batch\_size indicates the number of data records in each batch. Assume that each batch contains 32 data records. Modify the image size, normalization, and image channel, and then modify the data type of the label. Perform the shuffle operation, set batch\_size, and set drop\_remainder to True. In this case, data that cannot form a batch in the dataset will be discarded.

MindSpore supports multiple data processing and argumentation operations, which are usually used together. For details, see Data Processing and Data Argumentation.

Visualize samples.

Read the first 10 samples and visualize the samples to determine whether the samples are real datasets.

Code:

Display the first 10 images and the labels, and check whether the images are correctly labeled.

ds = create\_dataset(training=False)

data = ds.create\_dict\_iterator().\_\_next\_\_()

images = data['image'].asnumpy()

labels = data['label'].asnumpy()

plt.figure(figsize=(15,5))

for i in range(1,11):

plt.subplot(2, 5, i)

plt.imshow(np.squeeze(images[i]))

plt.title('Number: %s' % labels[i])

plt.xticks([])

plt.show()

Output:


Sample visualization

Define a network.

We define a simple fully-connected network to implement image recognition. The network has only three layers:

The first layer is a fully-connected layer in the shape of 784 x 512.

The second layer is a fully-connected layer in the shape of 512 x 128.

The last layer is an output layer in the shape of 128 x 10.

To use MindSpore for neural network definition, inherit mindspore.nn.cell.Cell. Cell is the base class of all neural networks (such as Conv2d).

Define each layer of a neural network in the \_\_init\_\_ method in advance, and then define the construct method to complete the forward construction of the neural network. The network layers are defined as follows:

Code:

Create a deep neural network (DNN) model. The model consists of three fully-connected layers. The final output layer uses softmax for classification (10 classes represented by numbers from 0 to 9)

class ForwardNN(nn.Cell):

def \_\_init\_\_(self):

super(ForwardNN, self).\_\_init\_\_()

self.flatten = nn.Flatten()

self.fc1 = nn.Dense(784, 512, activation='relu')

self.fc2 = nn.Dense(512, 128, activation='relu')

self.fc3 = nn.Dense(128, 10, activation=None)

def construct(self, input\_x):

output = self.flatten(input\_x)

output = self.fc1(output)

output = self.fc2(output)

output = self.fc3(output)

return output

Define a loss function and an optimizer.

A loss function is also called an objective function and is used to measure the difference between a predicted value and an actual value. Deep learning reduces the loss value by continuous iteration. Defining a good loss function can effectively improve the model performance.

An optimizer is used to minimize the loss function, improving the model during training.

After the loss function is defined, the weight-related gradient of the loss function can be obtained. The gradient is used to indicate the weight optimization direction for the optimizer, improving model performance. Loss functions supported by MindSpore include SoftmaxCrossEntropyWithLogits, L1Loss, and MSELoss.

SoftmaxCrossEntropyWithLogits is used in this example.

MindSpore provides the callback mechanism to execute custom logic during training. The following uses ModelCheckpoint provided by the framework as an example. ModelCheckpoint can save the network model and parameters for subsequent fine-tuning.

Code:

Create a network, a loss function, validation metric, and optimizer, and set related hyperparameters.

lr = 0.001

num\_epoch = 10

momentum = 0.9

net...