Shortcuts

mmedit.models.editors.iconvsr.iconvsr_net

Module Contents

Classes

IconVSRNet

IconVSR network structure for video super-resolution.

EDVRFeatureExtractor

EDVR feature extractor for information-refill in IconVSR.

class mmedit.models.editors.iconvsr.iconvsr_net.IconVSRNet(mid_channels=64, num_blocks=30, keyframe_stride=5, padding=2, spynet_pretrained=None, edvr_pretrained=None)[源代码]

Bases: mmengine.model.BaseModule

IconVSR network structure for video super-resolution.

Support only x4 upsampling.

Paper:

BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond, CVPR, 2021

参数
  • mid_channels (int) – Channel number of the intermediate features. Default: 64.

  • num_blocks (int) – Number of residual blocks in each propagation branch. Default: 30.

  • keyframe_stride (int) – Number determining the keyframes. If stride=5, then the (0, 5, 10, 15, …)-th frame will be the keyframes. Default: 5.

  • padding (int) – Number of frames to be padded at two ends of the sequence. 2 for REDS and 3 for Vimeo-90K. Default: 2.

  • spynet_pretrained (str) – Pre-trained model path of SPyNet. Default: None.

  • edvr_pretrained (str) – Pre-trained model path of EDVR (for refill). Default: None.

spatial_padding(lrs)[源代码]

Apply pdding spatially.

Since the PCD module in EDVR requires that the resolution is a multiple of 4, we apply padding to the input LR images if their resolution is not divisible by 4.

参数

lrs (Tensor) – Input LR sequence with shape (n, t, c, h, w).

返回

Padded LR sequence with shape (n, t, c, h_pad, w_pad).

返回类型

Tensor

check_if_mirror_extended(lrs)[源代码]

Check whether the input is a mirror-extended sequence.

If mirror-extended, the i-th (i=0, …, t-1) frame is equal to the (t-1-i)-th frame.

参数

lrs (tensor) – Input LR images with shape (n, t, c, h, w)

compute_refill_features(lrs, keyframe_idx)[源代码]

Compute keyframe features for information-refill.

Since EDVR-M is used, padding is performed before feature computation. :param lrs: Input LR images with shape (n, t, c, h, w) :type lrs: Tensor :param keyframe_idx: The indices specifying the keyframes. :type keyframe_idx: list(int)

返回

The keyframe features. Each key corresponds to the

indices in keyframe_idx.

返回类型

dict(Tensor)

compute_flow(lrs)[源代码]

Compute optical flow using SPyNet for feature warping.

Note that if the input is an mirror-extended sequence, ‘flows_forward’ is not needed, since it is equal to ‘flows_backward.flip(1)’.

参数

lrs (tensor) – Input LR images with shape (n, t, c, h, w)

返回

Optical flow. ‘flows_forward’ corresponds to the

flows used for forward-time propagation (current to previous). ‘flows_backward’ corresponds to the flows used for backward-time propagation (current to next).

返回类型

tuple(Tensor)

forward(lrs)[源代码]

Forward function for IconVSR.

参数

lrs (Tensor) – Input LR tensor with shape (n, t, c, h, w).

返回

Output HR tensor with shape (n, t, c, 4h, 4w).

返回类型

Tensor

class mmedit.models.editors.iconvsr.iconvsr_net.EDVRFeatureExtractor(in_channels=3, out_channel=3, mid_channels=64, num_frames=5, deform_groups=8, num_blocks_extraction=5, num_blocks_reconstruction=10, center_frame_idx=2, with_tsa=True, pretrained=None)[源代码]

Bases: mmengine.model.BaseModule

EDVR feature extractor for information-refill in IconVSR.

We use EDVR-M in IconVSR. To adopt pretrained models, please specify “pretrained”.

Paper: EDVR: Video Restoration with Enhanced Deformable Convolutional Networks. :param in_channels: Channel number of inputs. :type in_channels: int :param out_channels: Channel number of outputs. :type out_channels: int :param mid_channels: Channel number of intermediate features.

Default: 64.

参数
  • num_frames (int) – Number of input frames. Default: 5.

  • deform_groups (int) – Deformable groups. Defaults: 8.

  • num_blocks_extraction (int) – Number of blocks for feature extraction. Default: 5.

  • num_blocks_reconstruction (int) – Number of blocks for reconstruction. Default: 10.

  • center_frame_idx (int) – The index of center frame. Frame counting from 0. Default: 2.

  • with_tsa (bool) – Whether to use TSA module. Default: True.

  • pretrained (str) – The pretrained model path. Default: None.

forward(x)[源代码]

Forward function for EDVRFeatureExtractor.

参数

x (Tensor) – Input tensor with shape (n, t, 3, h, w).

返回

Intermediate feature with shape (n, mid_channels, h, w).

返回类型

Tensor

Read the Docs v: latest
Versions
master
latest
stable
zyh-doc-notfound-extend
Downloads
pdf
epub
On Read the Docs
Project Home
Builds

Free document hosting provided by Read the Docs.