Loading... ## 前言 Transformer模型源码解析过程参见前文, 这里是后续的实战示例. <div class="preview"> <div class="post-inser post box-shadow-wrap-normal"> <a href="https://zoe.red/2024/367.html" target="_blank" class="post_inser_a no-external-link no-underline-link"> <div class="inner-image bg" style="background-image: url(https://zoe.red/usr/uploads/2024/03/3146678007.png);background-size: cover;"></div> <div class="inner-content" > <p class="inser-title">Tranformer源码解读(Pytorch版本)</p> <div class="inster-summary text-muted"> 前言哈佛NLP团队实现的Pytorch版代码, 项目链接. 论文链接以下内容按照自己的理解过程一步步记录, 与源码... </div> </div> </a> <!-- .inner-content #####--> </div> <!-- .post-inser ####--> </div> ## 数据处理 ### 语料 这里使用对齐后的语料, 来自`Multi30k`数据集, 它依赖`torchtext`和`torchdata`2个库 * 安装脚本: `pip install -U torchtext torchdata` 使用示例 * 加载数据集, 并遍历获取文本/句子对 ```python # 指定数据集本地下载路径 # 每个数据集对象为一个迭代器对象,返回的内容为语言对元组: eg (德语, 英语) train, val, test = datasets.Multi30k(root='raw', language_pair=("de", "en")) # 遍历 train_text = [[], []] for pair_text in (train + val + test): train_text[0] += [pair_text[0]] train_text[1] += [pair_text[1]] ``` 可能问题: * test遍历过程, 提示解码相关'utf-8'错误 * 参见: https://github.com/pytorch/text/issues/2221, * 选择第二条解决问题, 修改源码_filter_fn函数返回代码 ![修改_filter_fn源码](https://zoe.red/usr/uploads/2024/03/945521704.png) ### 分词器 简单来说分词器是将输入文本分成一个个的词, 对于英文很好区分, 中文相对复杂些. 示例如下: ```plaintext # 英文 "mom, I love you." => ["mom", ",", "I", "love", "you", "."] "We like firework." => ["We", "like", "firework", "."] # 中文 "妈妈我爱你." => ["妈妈", "我", "爱", "你", "."]. "我们喜欢烟火." => ["我们", "喜欢", "烟火", "."] ``` `spacy`库提供了多种分词器, 方便大家直接调用. * 安装: `pip install spacy` * 下载: 第一次使用特定的分词器模型, 需要下载对应的分词器模型到本地 示例代码 ```python def load_tokenizers(): """ 使用spacy进行分词 spacy要求在虚拟环境中下载对应的分词语言模型, 以便于进行tokenize(分词) :return: 返回德语分词模型&英语分词模型 """ try: spacy_de = spacy.load("de_core_news_sm") except IOError: # 执行这里, 表示本地找不到对应的分词模型文件, 需要进行下载后再重新加载 os.system("python -m spacy download de_core_news_sm") spacy_de = spacy.load("de_core_news_sm") try: spacy_en = spacy.load("en_core_web_sm") except IOError: # 同上 os.system("python -m spacy download en_core_web_sm") spacy_en = spacy.load("en_core_web_sm") return spacy_de, spacy_en ``` ### 预处理与构建数据集 数据预处理工具类主要提供了: * 加载分词器 * 加载语料文本数据并结合分词器获取分词后结果 * 构建词汇表: * 4个特殊字符 * 全部分词后结果 实现代码 ```python class DatasetPreprocess: """数据集预处理类 初始化分词器 初始化语言数据集, 构造源语言与目标语言的字典, 句子长度范围 """ def __init__(self,): super().__init__() # 通过外部依次调用获取吧 # vocab_src, vocab_tgt, max_len_src, max_len_tgt = self.load_vocab() # train, val, test = datasets.Multi30k(root='raw', language_pair=("de", "en")) # spacy_de, spacy_en = self.load_tokenizers() def load_dataset_with_tokenize(self, spacy_de, spacy_en): """ 获取token化的数据集 """ train, val, test = datasets.Multi30k(root='raw', language_pair=("de", "en")) train, val, test = [ [(self.tokenize(sen_pair[0], spacy_de), (self.tokenize(sen_pair[1], spacy_en))) for sen_pair in dataset] for dataset in (train, val, test) ] return train, val, test @staticmethod def load_tokenizers(): """ 使用spacy进行分词 spacy要求在虚拟环境中下载对应的分词语言模型, 以便于进行tokenize(分词) :return: 返回德语分词模型&英语分词模型 """ try: spacy_de = spacy.load("de_core_news_sm") except IOError: # 执行这里, 表示本地找不到对应的分词模型文件, 需要进行下载后再重新加载 os.system("python -m spacy download de_core_news_sm") spacy_de = spacy.load("de_core_news_sm") try: spacy_en = spacy.load("en_core_web_sm") except IOError: # 同上 os.system("python -m spacy download en_core_web_sm") spacy_en = spacy.load("en_core_web_sm") return spacy_de, spacy_en def tokenize(self, text, tokenizer): """对text进行分词 Args: text (_type_): 要分词的文本, 例如 "I love you." tokenizer (_type_): 对应语言的分词模型, 例如: spacy_en Return: 分词结果,例如 ["I", "love", "you", "."] """ return [tok.text for tok in tokenizer.tokenizer(text)] @staticmethod def yield_tokens(data_iter, tokenizer, index): """生成器, 连续生成返回 token_list Args: data_iter (_type_): 包含句子对儿的可迭代对象。例如:[("I love you", "我爱你"), ...] tokenizer (_type_): 对应语言的分词模型。例如spacy_en index (_type_): 对应语言所在的索引位置, 对照data_iter的基本元素语言对顺序 Return: yield本轮的分词结果, 例如['I', 'love', 'you'] """ for from_to_tuple in data_iter: yield tokenizer(from_to_tuple[index]) def build_vocabulary(self, spacy_de, spacy_en): """ 构建字典/词汇表 统计数据集句子长度 """ def tokenize_de(text): return self.tokenize(text, spacy_de) def tokenize_en(text): return self.tokenize(text, spacy_en) def get_sentence_max_length(data_iter): """获取数据集的语句长度 token个数 Args: data_iter (_type_): 全部数据的合集 Returns: int: 最长句子的token个数 """ return max([len(_) for _ in data_iter]) print("创建源语言词典: German ...") # 最新版库, test数据集 解析数据提示编码错误 # 参见: https://github.com/pytorch/text/issues/2221 # 根据最后的回答, 选择第二条解决问题, 修改源码_filter_fn函数返回代码 train, val, test = datasets.Multi30k(root='raw', language_pair=("de", "en")) vocab_src = build_vocab_from_iterator( self.yield_tokens(train + val + test , tokenize_de, index=0), min_freq=2, specials=["<s>", "</s>", "<pad>", "<unk>"], # specials=["<pad>", "<unk>", "<s>", "</s>"], special_first=True, # 构建词汇表时会在之前 ) # print("创建目标语言词典: English ...") # train, val, test = datasets.Multi30k(root='raw', language_pair=("de", "en")) # vocab_tgt = build_vocab_from_iterator( # self.yield_tokens(train + val + test, tokenize_en, index=1), # # specials=["<pad>", "<s>", "</s>", "<unk>"], # specials=["<s>", "</s>", "<pad>", "<unk>"], # special_first=True, # 构建词汇表时会在之前 # ) print("Building English Vocabulary ...") train, val, test = datasets.Multi30k(root='raw', language_pair=("de", "en")) vocab_tgt = build_vocab_from_iterator( self.yield_tokens(train + val + test, tokenize_en, index=1), min_freq=2, specials=["<s>", "</s>", "<pad>", "<unk>"], # specials=["<pad>", "<unk>", "<s>", "</s>"], special_first=True, # 构建词汇表时会在之前 ) # 如果不在词汇表中, 即未知单词, 直接当做<unk> vocab_src.set_default_index(vocab_src["<unk>"]) vocab_tgt.set_default_index(vocab_tgt["<unk>"]) # # 扩展 max_len_src = get_sentence_max_length(self.yield_tokens(train + val + test, tokenize_de, index=0)) max_len_tgt = get_sentence_max_length(self.yield_tokens(train + val + test, tokenize_en, index=1)) return vocab_src, vocab_tgt, max_len_src, max_len_tgt def load_vocab(self, spacy_de, spacy_en): vocat_path = "vocab_my.pt" if not os.path.exists(vocat_path): vocab_src, vocab_tgt, max_len_src, max_len_tgt = self.build_vocabulary(spacy_de, spacy_en) torch.save((vocab_src, vocab_tgt, max_len_src, max_len_tgt), vocat_path) else: vocab_src, vocab_tgt, max_len_src, max_len_tgt = torch.load(vocat_path) print("Finished.\n字典长度:") print('源语言:', len(vocab_src), '句子最大长度:', max_len_src) print('目标语言:', len(vocab_tgt), '句子最大长度:', max_len_tgt) return vocab_src, vocab_tgt, max_len_src, max_len_tgt ``` 数据集定义 ```python class LanguageDataset(Dataset): """说明 批量训练评测数据构造, """ def __init__(self, dataset, vocab_src, vocab_tgt, max_len_src, max_len_tgt): """_summary_ Args: dataset (_type_): 数据集(翻译双语) vocab_src (_type_): SRC词汇表对象 vocab_tgt (_type_): TGT词汇表对象 max_len_src (_type_): SRC语句最大长度 max_len_tgt (_type_): TGT语句最大长度 """ super().__init__() self.dataset = dataset self.vocab_src = vocab_src self.vocab_tgt = vocab_tgt self.max_len_src = max_len_src self.max_len_tgt = max_len_tgt # self.sentence_length_src = max_len_src + 1 # 每个句子末尾都放上结束标记字符 # self.sentence_length_tgt = max_len_tgt + 2 # 每个句子开头结尾都加上标记 # 每个句子开头结尾都加上标记 self.sentence_length_src = max_len_src self.sentence_length_tgt = max_len_tgt def __getitem__(self, index): # 分词后的句子 src, tgt = self.dataset[index] # 映射到词典&统一长度 src = [self.vocab_src['<s>']] + self.vocab_src(src) + [self.vocab_src['</s>']] + \ ([self.vocab_src['<pad>']]*(self.sentence_length_src - 1 - len(src))) src = torch.from_numpy(np.array(src)) tgt = [self.vocab_tgt['<s>']] + self.vocab_tgt(tgt) + [self.vocab_tgt['</s>']] + \ ([self.vocab_tgt['<pad>']]*(self.sentence_length_tgt - 1 - len(tgt))) tgt = torch.from_numpy(np.array(tgt)) return src, tgt def __len__(self): return len(self.dataset) ``` 实例化一个dataloader ```python class Batch: """Object for holding a batch of data with mask during training. """ def __init__(self, src, tgt=None, pad=2): self.src = src self.src_mask = (src != pad).unsqueeze(-2) # [B,Sentence] => [B, 1, Sentence] if tgt is not None: self.tgt = tgt[:, :-1] # 目标句子的[0, n-1], 作为decoder输入 self.tgt_y = tgt[:, 1:] # 目标句子的[1, n], 作为预测输出的GT self.tgt_mask = self.make_std_mask(self.tgt, pad) self.ntokens = (self.tgt_y != pad).data.sum() @staticmethod def make_std_mask(tgt, pad): "Create a mask to hide padding and future words." tgt_mask = (tgt != pad).unsqueeze(-2) # [B,Sentence-1] => [B, 1, Sentence-1] # [B, 1, Sentence-1] & [1, Sentence-1, Sentence-1] => [B, Sentence-1, Sentence-1] # 解释: 2个张量分别被广播, 变为[B, Sentence-1, Sentence-1]之后再运算 tgt_mask = tgt_mask & subsequent_mask(tgt.size(-1)).type_as(tgt_mask.data) return tgt_mask def get_collate_fn(pad_id): def collate_fn(batch): """ 输入为list, 为batch_size个dataset的__getitem__ 的返回结果 """ tensor_src = torch.stack([_[0] for _ in batch], dim=0) tensor_tgt = torch.stack([_[1] for _ in batch], dim=0) batch = Batch(tensor_src, tensor_tgt, pad=pad_id) return batch return collate_fn # init dp = DatasetPreprocess() spacy_de, spacy_en = dp.load_tokenizers() train, val, test = dp.load_dataset_with_tokenize(spacy_de, spacy_en) vocab_src, vocab_tgt, max_len_src, max_len_tgt = dp.load_vocab(spacy_de, spacy_en) max_len_src, max_len_tgt = 72, 72 print('dataset init finished...') # 词汇表中的4个特殊字符定义, 在源/目标词汇表中的映射是一致的 pad_str = '<pad>' pad_id = vocab_src[pad_str] collate_fn = get_collate_fn(pad_id=pad_idx) train_dataloader = DataLoader( ds_train, batch_size=config.batch_size, shuffle=True, num_workers=config.num_workers, collate_fn=collate_fn, drop_last=True, ) ``` --- ## 训练推理 *训练损失与优化器* ```python criterion = LabelSmoothing(size=configs.vocab, padding_idx=0, smoothing=0.1) optimizer = torch.optim.Adam(model.parameters(), lr=1e-3, betas=(0.9, 0.98), eps=1e-9) ``` *学习率策略:* * warmup方式, 学习率一开始很小逐渐增大,之后再下载, 每个batch后都会更新一次学习率 * batchsize=32, 此处尝试变大batchsize对于warmup的学习效果影响, 发现变差了(最后的验证集loss变大) *完整代码链接:* * Github链接: 待补充... *输出日志* * 贪心解码 预测输出 ```plaintext CPU信息: 物理核心: 12, 线程数: 24, 使用率: 0.0% 型号: AMD Ryzen 9 3900X 12-Core Processor RAM信息: 全部: 64,237.3 MB, 空闲: 60,477.1 MB, 全部使用: 3,038.3 MB, 当前进程使用: 433.1 MB =========================== GPU数量: 2 =========================== 驱动: v550.54.15 GPU-0 Model: NVIDIA GeForce RTX 4060 Ti; VRAM: total=16380.0 MB, free=16052.1 MB, used=327.9 MB GPU-1 Model: NVIDIA GeForce RTX 4060 Ti; VRAM: total=16380.0 MB, free=16064.8 MB, used=315.2 MB PyTorch 版本: 2.2.2+cu121 CUDA 可用性: True CUDA 版本: 12.1 cuDNN 版本: 8902 创建源语言词典: German ... Building English Vocabulary ... Finished. 字典长度: 源语言: 8316 句子最大长度: 44 目标语言: 6384 句子最大长度: 41 dataset init finished... ***************************** 全局参数 ***************************** base_lr: 1.0 warmup: 3000 batch_size: 32 num_epochs: 8 interval_eval: 1 accum_iter: 10 file_prefix: multi30k_model_ max_len_src: 72 max_len_tgt: 72 vocab_src: Vocab() vocab_tgt: Vocab() pad_id: 2 embed_dim: 512 N: 6 num_workers: 4 gpu_ids: [0] 模型参数位置: cuda:0 ************************** epoch=01/8 ************************** train ... 24/03/29 23:36:15 Epoch Step: 040/906 | Accumulation Step: 004 | Loss: 7.50 | Tokens / Sec: 4573.3 | Learning Rate: 0.00001076 24/03/29 23:36:18 Epoch Step: 080/906 | Accumulation Step: 008 | Loss: 7.09 | Tokens / Sec: 5095.8 | Learning Rate: 0.00002152 24/03/29 23:36:22 Epoch Step: 120/906 | Accumulation Step: 012 | Loss: 6.79 | Tokens / Sec: 5089.5 | Learning Rate: 0.00003227 24/03/29 23:36:26 Epoch Step: 160/906 | Accumulation Step: 016 | Loss: 6.54 | Tokens / Sec: 5075.9 | Learning Rate: 0.00004303 24/03/29 23:36:29 Epoch Step: 200/906 | Accumulation Step: 020 | Loss: 6.36 | Tokens / Sec: 5086.0 | Learning Rate: 0.00005379 24/03/29 23:36:33 Epoch Step: 240/906 | Accumulation Step: 024 | Loss: 6.25 | Tokens / Sec: 5087.1 | Learning Rate: 0.00006455 24/03/29 23:36:37 Epoch Step: 280/906 | Accumulation Step: 028 | Loss: 6.13 | Tokens / Sec: 5086.4 | Learning Rate: 0.00007531 24/03/29 23:36:40 Epoch Step: 320/906 | Accumulation Step: 032 | Loss: 5.88 | Tokens / Sec: 4957.4 | Learning Rate: 0.00008607 24/03/29 23:36:44 Epoch Step: 360/906 | Accumulation Step: 036 | Loss: 5.63 | Tokens / Sec: 5014.5 | Learning Rate: 0.00009682 24/03/29 23:36:47 Epoch Step: 400/906 | Accumulation Step: 040 | Loss: 5.54 | Tokens / Sec: 5065.2 | Learning Rate: 0.00010758 24/03/29 23:36:51 Epoch Step: 440/906 | Accumulation Step: 044 | Loss: 5.21 | Tokens / Sec: 5045.7 | Learning Rate: 0.00011834 24/03/29 23:36:55 Epoch Step: 480/906 | Accumulation Step: 048 | Loss: 4.97 | Tokens / Sec: 4998.3 | Learning Rate: 0.00012910 24/03/29 23:36:58 Epoch Step: 520/906 | Accumulation Step: 052 | Loss: 4.97 | Tokens / Sec: 5030.3 | Learning Rate: 0.00013986 24/03/29 23:37:02 Epoch Step: 560/906 | Accumulation Step: 056 | Loss: 4.78 | Tokens / Sec: 4970.6 | Learning Rate: 0.00015062 24/03/29 23:37:06 Epoch Step: 600/906 | Accumulation Step: 060 | Loss: 4.59 | Tokens / Sec: 5146.5 | Learning Rate: 0.00016137 24/03/29 23:37:09 Epoch Step: 640/906 | Accumulation Step: 064 | Loss: 4.45 | Tokens / Sec: 5019.4 | Learning Rate: 0.00017213 24/03/29 23:37:13 Epoch Step: 680/906 | Accumulation Step: 068 | Loss: 4.30 | Tokens / Sec: 4998.3 | Learning Rate: 0.00018289 24/03/29 23:37:17 Epoch Step: 720/906 | Accumulation Step: 072 | Loss: 4.15 | Tokens / Sec: 5038.4 | Learning Rate: 0.00019365 24/03/29 23:37:20 Epoch Step: 760/906 | Accumulation Step: 076 | Loss: 4.32 | Tokens / Sec: 4932.5 | Learning Rate: 0.00020441 24/03/29 23:37:24 Epoch Step: 800/906 | Accumulation Step: 080 | Loss: 4.14 | Tokens / Sec: 5034.4 | Learning Rate: 0.00021517 24/03/29 23:37:28 Epoch Step: 840/906 | Accumulation Step: 084 | Loss: 4.21 | Tokens / Sec: 5041.2 | Learning Rate: 0.00022592 24/03/29 23:37:31 Epoch Step: 880/906 | Accumulation Step: 088 | Loss: 3.95 | Tokens / Sec: 4996.8 | Learning Rate: 0.00023668 24/03/29 23:37:34 Epoch Step: 906/906 | Accumulation Step: 091 | Loss: 3.86 | Tokens / Sec: 5021.0 | Learning Rate: 0.00024368 函数 run_epoch 执行时间: 82.9 s. valid... 函数 run_epoch 执行时间: 1.2 s. tensor(3.917813) ... ************************** epoch=08/8 ************************** train ... 24/03/29 23:46:15 Epoch Step: 040/906 | Accumulation Step: 004 | Loss: 1.37 | Tokens / Sec: 4860.1 | Learning Rate: 0.00055321 24/03/29 23:46:19 Epoch Step: 080/906 | Accumulation Step: 008 | Loss: 1.35 | Tokens / Sec: 4998.5 | Learning Rate: 0.00055148 24/03/29 23:46:23 Epoch Step: 120/906 | Accumulation Step: 012 | Loss: 1.28 | Tokens / Sec: 4989.8 | Learning Rate: 0.00054977 24/03/29 23:46:26 Epoch Step: 160/906 | Accumulation Step: 016 | Loss: 1.42 | Tokens / Sec: 5028.3 | Learning Rate: 0.00054808 24/03/29 23:46:30 Epoch Step: 200/906 | Accumulation Step: 020 | Loss: 1.37 | Tokens / Sec: 5050.8 | Learning Rate: 0.00054640 24/03/29 23:46:34 Epoch Step: 240/906 | Accumulation Step: 024 | Loss: 1.02 | Tokens / Sec: 5059.8 | Learning Rate: 0.00054474 24/03/29 23:46:37 Epoch Step: 280/906 | Accumulation Step: 028 | Loss: 1.11 | Tokens / Sec: 5048.7 | Learning Rate: 0.00054309 24/03/29 23:46:41 Epoch Step: 320/906 | Accumulation Step: 032 | Loss: 1.10 | Tokens / Sec: 5038.7 | Learning Rate: 0.00054146 24/03/29 23:46:44 Epoch Step: 360/906 | Accumulation Step: 036 | Loss: 1.38 | Tokens / Sec: 4954.3 | Learning Rate: 0.00053984 24/03/29 23:46:48 Epoch Step: 400/906 | Accumulation Step: 040 | Loss: 1.21 | Tokens / Sec: 4948.6 | Learning Rate: 0.00053823 24/03/29 23:46:52 Epoch Step: 440/906 | Accumulation Step: 044 | Loss: 1.24 | Tokens / Sec: 5035.1 | Learning Rate: 0.00053664 24/03/29 23:46:55 Epoch Step: 480/906 | Accumulation Step: 048 | Loss: 1.18 | Tokens / Sec: 5025.6 | Learning Rate: 0.00053507 24/03/29 23:46:59 Epoch Step: 520/906 | Accumulation Step: 052 | Loss: 1.02 | Tokens / Sec: 5031.4 | Learning Rate: 0.00053351 24/03/29 23:47:03 Epoch Step: 560/906 | Accumulation Step: 056 | Loss: 1.33 | Tokens / Sec: 5056.4 | Learning Rate: 0.00053196 24/03/29 23:47:06 Epoch Step: 600/906 | Accumulation Step: 060 | Loss: 1.47 | Tokens / Sec: 4945.5 | Learning Rate: 0.00053042 24/03/29 23:47:10 Epoch Step: 640/906 | Accumulation Step: 064 | Loss: 1.20 | Tokens / Sec: 5028.9 | Learning Rate: 0.00052890 24/03/29 23:47:14 Epoch Step: 680/906 | Accumulation Step: 068 | Loss: 1.21 | Tokens / Sec: 5065.7 | Learning Rate: 0.00052739 24/03/29 23:47:17 Epoch Step: 720/906 | Accumulation Step: 072 | Loss: 1.25 | Tokens / Sec: 5061.7 | Learning Rate: 0.00052590 24/03/29 23:47:21 Epoch Step: 760/906 | Accumulation Step: 076 | Loss: 1.39 | Tokens / Sec: 5078.1 | Learning Rate: 0.00052441 24/03/29 23:47:25 Epoch Step: 800/906 | Accumulation Step: 080 | Loss: 1.29 | Tokens / Sec: 4958.5 | Learning Rate: 0.00052294 24/03/29 23:47:28 Epoch Step: 840/906 | Accumulation Step: 084 | Loss: 1.43 | Tokens / Sec: 5021.0 | Learning Rate: 0.00052149 24/03/29 23:47:32 Epoch Step: 880/906 | Accumulation Step: 088 | Loss: 1.19 | Tokens / Sec: 5099.3 | Learning Rate: 0.00052004 24/03/29 23:47:34 Epoch Step: 906/906 | Accumulation Step: 091 | Loss: 1.19 | Tokens / Sec: 5069.5 | Learning Rate: 0.00051911 函数 run_epoch 执行时间: 82.9 s. valid... 函数 run_epoch 执行时间: 1.2 s. tensor(1.563832) 加载权重初始化: eval_loss=1.5638318 模型参数位置: cuda:0 ---------------------------复现验证集评估结果---------------------------- 函数 run_epoch 执行时间: 1.2 s. (tensor(1.563832, device='cuda:0'), <__main__.TrainState object at 0x79f045642890>) -------------------------Batch_Index=0-------------------------- sentence_0 src =['<s>', 'Ein', 'Mann', 'mit', 'einem', 'orangefarbenen', 'Hut', ',', 'der', 'etwas', '<unk>', '.', '</s>'] sentence_0 tgt =['<s>', 'A', 'man', 'in', 'an', 'orange', 'hat', 'starring', 'at', 'something', '.', '</s>'] sentence_0 pre =['<s>', 'A', 'man', 'in', 'an', 'orange', 'hat', ',', '<unk>', 'something', '.', '</s>'] sentence_1 src =['<s>', 'Ein', 'Boston', 'Terrier', 'läuft', 'über', '<unk>', 'Gras', 'vor', 'einem', 'weißen', 'Zaun', '.', '</s>'] sentence_1 tgt =['<s>', 'A', 'Boston', 'Terrier', 'is', 'running', 'on', 'lush', 'green', 'grass', 'in', 'front', 'of', 'a', 'white', 'fence', '.', '</s>'] sentence_1 pre =['<s>', 'A', '<unk>', 'dog', 'runs', 'across', 'the', 'grass', 'in', 'front', 'of', 'a', 'white', 'fence', '.', '</s>'] sentence_2 src =['<s>', 'Ein', 'Mädchen', 'in', 'einem', 'Karateanzug', 'bricht', 'ein', 'Brett', 'mit', 'einem', 'Tritt', '.', '</s>'] sentence_2 tgt =['<s>', 'A', 'girl', 'in', 'karate', 'uniform', 'breaking', 'a', 'stick', 'with', 'a', 'front', 'kick', '.', '</s>'] sentence_2 pre =['<s>', 'A', 'girl', 'in', 'a', 'karate', 'uniform', 'is', 'shooting', 'a', 'board', 'with', 'a', 'man', 'in', 'a', 'uniform', '.', '</s>'] sentence_3 src =['<s>', 'Fünf', 'Leute', 'in', 'Winterjacken', 'und', 'mit', 'Helmen', 'stehen', 'im', 'Schnee', 'mit', '<unk>', 'im', 'Hintergrund', '.', '</s>'] sentence_3 tgt =['<s>', 'Five', 'people', 'wearing', 'winter', 'jackets', 'and', 'helmets', 'stand', 'in', 'the', 'snow', ',', 'with', '<unk>', 'in', 'the', 'background', '.', '</s>'] sentence_3 pre =['<s>', 'Five', 'people', 'in', 'winter', 'jackets', 'and', 'helmets', 'are', 'standing', 'in', 'the', 'snow', 'with', '<unk>', 'in', 'the', 'background', '.', '</s>'] sentence_4 src =['<s>', 'Leute', 'Reparieren', 'das', 'Dach', 'eines', 'Hauses', '.', '</s>'] sentence_4 tgt =['<s>', 'People', 'are', 'fixing', 'the', 'roof', 'of', 'a', 'house', '.', '</s>'] sentence_4 pre =['<s>', 'People', 'are', 'walking', 'on', 'the', 'roof', 'of', 'a', 'house', '.', '</s>'] sentence_5 src =['<s>', 'Ein', 'hell', 'gekleideter', 'Mann', 'fotografiert', 'eine', 'Gruppe', 'von', 'Männern', 'in', 'dunklen', 'Anzügen', 'und', 'mit', 'Hüten', ',', 'die', 'um', 'eine', 'Frau', 'in', 'einem', 'trägerlosen', 'Kleid', 'herum', 'stehen', '.', '</s>'] sentence_5 tgt =['<s>', 'A', 'man', 'in', 'light', 'colored', 'clothing', 'photographs', 'a', 'group', 'of', 'men', 'wearing', 'dark', 'suits', 'and', 'hats', 'standing', 'around', 'a', 'woman', 'dressed', 'in', 'a', '<unk>', 'gown', '.', '</s>'] sentence_5 pre =['<s>', 'A', 'well', 'dressed', 'man', 'dressed', 'in', 'dark', 'clothes', 'and', 'a', 'group', 'of', 'men', 'in', 'dark', 'hats', ',', 'hats', ',', 'are', 'standing', 'around', 'a', 'table', 'in', 'a', 'museum', '.', '</s>'] sentence_6 src =['<s>', 'Eine', 'Gruppe', 'von', 'Menschen', 'steht', 'vor', 'einem', 'Iglu', '.', '</s>'] sentence_6 tgt =['<s>', 'A', 'group', 'of', 'people', 'standing', 'in', 'front', 'of', 'an', 'igloo', '.', '</s>'] sentence_6 pre =['<s>', 'A', 'group', 'of', 'people', 'are', 'standing', 'in', 'front', 'of', 'an', 'amusement', '.', '</s>'] sentence_7 src =['<s>', 'Ein', 'Junge', 'in', 'einem', 'roten', 'Trikot', 'versucht', ',', 'die', 'Home', 'Base', 'zu', 'erreichen', ',', 'während', 'der', 'Catcher', 'im', 'blauen', 'Trikot', 'versucht', ',', 'ihn', 'zu', 'fangen', '.', '</s>'] sentence_7 tgt =['<s>', 'A', 'boy', 'in', 'a', 'red', 'uniform', 'is', 'attempting', 'to', 'avoid', 'getting', 'out', 'at', 'home', 'plate', ',', 'while', 'the', 'catcher', 'in', 'the', 'blue', 'uniform', 'is', 'attempting', 'to', 'catch', 'him', '.', '</s>'] sentence_7 pre =['<s>', 'A', 'boy', 'in', 'a', 'red', 'uniform', 'tries', 'to', 'get', 'ready', 'to', 'get', 'ready', 'to', 'get', 'the', 'base', 'while', 'the', 'other', 'in', 'a', 'blue', 'uniform', '.', '</s>'] ``` THE END 本文作者:将夜 本文链接:https://zoe.red/2024/376.html 版权声明:本博客所有文章除特别声明外,均默认采用 CC BY-NC-SA 4.0 许可协议。 最后修改:2024 年 03 月 29 日 © 允许规范转载 赞 如果觉得我的文章对你有用,请随意赞赏