Tutorial 1: Encoding histology image features using H-ViT¶

This tutorial shows how to apply H-ViT to extract the latent representation of each spot image. Here we take the slice 151674 of the DLPFC dataset as example. The histology image is available at https://github.com/LieberInstitute/spatialLIBD. To reduce your waiting time, we upload obtained histological image representation data to https://zenodo.org/record/8185216/files/DLPFC.rar?download=1.

Preparation¶

[1]:

import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import os
from stGCL.image_processing import image_representation
import os
import torch
import random
import numpy as np
from torch.backends import cudnn
import pandas as pd
from stGCL.process import set_seed

[2]:

# seed=0
# set_seed(seed)
section_id="151676"
pca_num = 3000

[3]:

labeldf = pd.read_csv("/home/dell/stproject/stGCL/Data/DLPFC/{}/cluster_labels_{}.csv".format(section_id, section_id), sep=",", header=0,
                      na_filter=False, index_col=0)
label = labeldf["ground_truth"]

[4]:

positionsdata = pd.read_csv("/home/dell/stproject/stGCL/Data/DLPFC/{}/spatial/tissue_positions_list.csv".format(section_id), sep=",", header=None,
                            na_filter=False, index_col=0)
positionsdata = positionsdata.loc[positionsdata[1] == 1].iloc[: , 3:5]
positionsdata.columns = ['x','y']

[5]:

scale_file="/home/dell/stproject/stGCL/Data/DLPFC/{}/spatial/scalefactors_json.json".format(section_id)
image_adress="/home/dell/stproject/stGCL/Data/DLPFC/{}/spatial".format(section_id)
score_adress="/home/dell/stproject/stGCL/image_representation/".format(section_id)
image_file="tissue_hires_image.png"

Extract histology image features¶

[6]:

image_representation(positionsdata,image_adress,image_file,label,score_adress,pca_num=3000,k=7,scale_file=scale_file,
                     image_net="ViT",image_type="hires",crop_size=256,patch_size=64,load=False,batch_size_lw=128)

Loading image data...

Tiling image: 100%|████████████████████████████████████████████████████████████████████████████████ [ time left: 00:00 ]
extract representation by ViT: 100%|████████████████████████████████████████████████████| 28/28 [00:09<00:00,  2.98it/s]

PCA dimensionality reduction
Image features are stored in:  /home/dell/stproject/stGCL/image_representation//ViT_representation.csv
PCA Image features are stored in:  /home/dell/stproject/stGCL/image_representation//ViT_pca_representation.csv
Complete the extraction of image representation