LLM-FineTuning-Large-Language-Models
/
BERT_HuggingFace_Basic_Usages.ipynb
3843 строки · 149.9 Кб
1{2"cells": [3{4"cell_type": "markdown",5"metadata": {6"id": "zs9Je-MNINnR"7},8"source": [9"## First What is BERT?\n",10"\n",11"BERT stands for Bidirectional Encoder Representations from Transformers. The name itself gives us several clues to what BERT is all about.\n",12"\n",13"BERT architecture consists of several Transformer encoders stacked together. Each Transformer encoder encapsulates two sub-layers: a self-attention layer and a feed-forward layer.\n",14"\n",15"### There are two different BERT models:\n",16"\n",17"- BERT base, which is a BERT model consists of 12 layers of Transformer encoder, 12 attention heads, 768 hidden size, and 110M parameters.\n",18"\n",19"- BERT large, which is a BERT model consists of 24 layers of Transformer encoder,16 attention heads, 1024 hidden size, and 340 parameters.\n",20"\n",21"\n",22"\n",23"BERT Input and Output\n",24"BERT model expects a sequence of tokens (words) as an input. In each sequence of tokens, there are two special tokens that BERT would expect as an input:\n",25"\n",26"- [CLS]: This is the first token of every sequence, which stands for classification token.\n",27"- [SEP]: This is the token that makes BERT know which token belongs to which sequence. This special token is mainly important for a next sentence prediction task or question-answering task. If we only have one sequence, then this token will be appended to the end of the sequence.\n",28"\n",29"\n",30"It is also important to note that the maximum size of tokens that can be fed into BERT model is 512. If the tokens in a sequence are less than 512, we can use padding to fill the unused token slots with [PAD] token. If the tokens in a sequence are longer than 512, then we need to do a truncation.\n",31"\n",32"And that’s all that BERT expects as input.\n",33"\n",34"BERT model then will output an embedding vector of size 768 in each of the tokens. We can use these vectors as an input for different kinds of NLP applications, whether it is text classification, next sentence prediction, Named-Entity-Recognition (NER), or question-answering.\n",35"\n",36"\n",37"------------\n",38"\n",39"**For a text classification task**, we focus our attention on the embedding vector output from the special [CLS] token. This means that we’re going to use the embedding vector of size 768 from [CLS] token as an input for our classifier, which then will output a vector of size the number of classes in our classification task.\n",40"\n",41"-----------------------\n",42"\n",43"\n",44"\n",45"-------------------------\n",46"\n",47"The core part of BERT is the stacked bidirectional encoders from the transformer model, but during pre-training, a masked language modeling and next sentence prediction head are added onto BERT.\n",48"\n",49"When I say “head”, I mean that a few extra layers are added onto BERT that can be used to generate a specific output. The raw output of BERT is the output from the stacked Bi-directional encoders.\n",50"\n",51"### Tokenization for Bert Models\n",52"\n",53"Tokenization plays an essential role in NLP as it helps convert the text to numbers which deep learning models can use for processing.\n",54"\n",55"No deep learning models can work directly with the text. You need to convert it into numbers or the format which the model can understand."56]57},58{59"cell_type": "code",60"execution_count": null,61"metadata": {62"colab": {63"base_uri": "https://localhost:8080/",64"height": 685,65"referenced_widgets": [66"3df2c3cfc54647ac9e206ea8287e486e",67"8783e4a95be0490990f30a9a7c750a68",68"6a39bd415eb346289c51fd067cb18700",69"e67f6c44a3424205815d92dcebfbd609",70"d4bf69ad6ca74580a4b35f115c029295",71"be6f639e74244154b1e21610f5249fbd",72"a129b4bb0cca4aae90a5ff0ee73926e3",73"ff9c374a59f1400aa0dc26d244ee85dd",74"505b5ad2473c4804b6876233f2357b67",75"908c9ba59b674a028013c41dcaa9b166",76"96ea79cbc9ec45889d2016d619195168",77"5439561e53d946ea88df73ea64e30e9d",78"ca3ab0fc2b304864bee1ce96be875d6c",79"4d467784792d4761a5e17fcc7025eafd",80"7455a72139d8459f8d9571173c20077c",81"73a2321c0f274add8e77cad80603580a",82"a8e763d06bb6456a8d9c62dcfc06111b",83"c70efd624f97461086e67dad24397742",84"3108d55dcc9b407bac67cb8abcc4a2c0",85"0472e24144684e1e8cb96e015a74a379",86"b5f908703c444e2d8150adfc755283a7",87"aeb4990bc4564386b09ca09bffe423fb",88"93b184b6eb584cb98ec731dab7ee81a9",89"3f1df1a788224fc589970e4b70b247fd",90"9537632b2a79434bac79702ea12bff68",91"bbb1759e97dd4c6c97015014503dc681",92"b76f3788d7b5486b8882996076912186",93"9005579c02fd4b00b3c935df9deae17f",94"7a54636d8b7f4e59bf12ecdd97a8c89a",95"6cc7360fd6e54b29829722243450869a",96"8bee921f44d34d709ea0827874e4ac63",97"017d3ebfbf8a46efa1e956bf8f358636",98"b1b0b418cabf43309e78cad077e14a72"99]100},101"id": "Ox-QTZa31D6u",102"outputId": "7b86ece5-5786-4ced-822a-77bdd8dda18a"103},104"outputs": [],105"source": [106"!pip install transformers\n",107"from transformers import BertTokenizer\n",108"tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')"109]110},111{112"cell_type": "code",113"execution_count": 2,114"metadata": {115"colab": {116"base_uri": "https://localhost:8080/"117},118"id": "WnfvaKxj1cs9",119"outputId": "bdc1fe09-5235-4aa0-d173-f4d28c04abc5"120},121"outputs": [122{123"data": {124"text/plain": [125"{'input_ids': tensor([[ 101, 16798, 2475, 2097, 2022, 1037, 2307, 2095, 2005, 2035,\n",126" 1997, 2149, 102, 0, 0, 0, 0, 0, 0, 0,\n",127" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",128" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",129" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",130" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",131" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",132" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",133" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",134" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",135" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",136" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",137" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",138" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",139" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",140" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",141" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",142" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",143" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",144" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",145" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",146" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",147" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",148" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",149" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",150" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",151" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",152" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",153" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",154" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",155" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",156" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",157" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",158" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",159" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",160" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",161" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",162" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",163" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",164" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",165" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",166" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",167" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",168" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",169" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",170" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",171" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",172" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",173" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",174" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",175" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",176" 0, 0]]), 'token_type_ids': tensor([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",177" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",178" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",179" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",180" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",181" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",182" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",183" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",184" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",185" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",186" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",187" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",188" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",189" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",190" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",191" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",192" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",193" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",194" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",195" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",196" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",197" 0, 0, 0, 0, 0, 0, 0, 0]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",198" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",199" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",200" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",201" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",202" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",203" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",204" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",205" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",206" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",207" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",208" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",209" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",210" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",211" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",212" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",213" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",214" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",215" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",216" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",217" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",218" 0, 0, 0, 0, 0, 0, 0, 0]])}"219]220},221"execution_count": 2,222"metadata": {},223"output_type": "execute_result"224}225],226"source": [227"text = '2022 will be a great year for all of us'\n",228"encoding = tokenizer.encode_plus(text, add_special_tokens = True, truncation = True, padding = \"max_length\", return_attention_mask = True, return_tensors = \"pt\")\n",229"encoding"230]231},232{233"cell_type": "markdown",234"metadata": {235"id": "lppcI2PfINnY"236},237"source": [238"**`padding = \"max_length\"`** => The padding argument controls padding. padded by zeros to make all the text to the length of `max_length`. So, it will pad to a length specified by the `max_length` argument or the maximum length accepted by the model if no max_length is provided. Padding will still be applied if you only provide a single sequence.\n",239"\n",240"**truncation = True** => True or 'longest_first': truncate to a maximum length specified by the max_length argument or the maximum length accepted by the model if no max_length is provided (max_length=None). This will truncate token by token, removing a token from the longest sequence in the pair until the proper length is reached.\n",241"\n",242"As BERT can only accept as input only 512 tokens at a time, so we must specify the truncation parameter to True. \n",243"\n",244"The **`add_special_tokens`** parameter is just for BERT to add tokens like the start, end, [SEP], and [CLS] tokens. `Return_tensors = “pt”` is just for the tokenizer to return PyTorch tensors. If you don’t want this to happen(maybe you want it to return a list), then you can remove the parameter and it will return lists.\n",245"\n",246"**tokenizer.encode_plus()** specifically returns a dictionary of values instead of just a list of values. Because tokenizer.encode_plus() can return many different types of information, like the attention_masks and token type ids, everything is returned in a dictionary format, and if you want to retrieve the specific parts of the encoding, you can do it like below"247]248},249{250"cell_type": "code",251"execution_count": 3,252"metadata": {253"id": "mqBlnJiw1cpR"254},255"outputs": [],256"source": [257"input = encoding[\"input_ids\"][0]\n",258"attention_mask = encoding[\"attention_mask\"][0]"259]260},261{262"cell_type": "markdown",263"metadata": {264"id": "iZGRYZtzINnZ"265},266"source": [267"Additionally, because the tokenizer returns a dictionary of different values, instead of finding those values as shown above and individually passing these into the model, we can just pass in the entire encoding like this\n",268"\n",269"```\n",270"output = model(**encoding)\n",271"```\n",272"\n",273"\n",274"One more very important thing about the tokenizer to know is that you can specify to retrieve specific tokens if desired. For example, if you are doing masked language modeling and you want to insert a mask at a location for your model to decode, then you can simply retrieve the mask token like this"275]276},277{278"cell_type": "code",279"execution_count": 4,280"metadata": {281"colab": {282"base_uri": "https://localhost:8080/",283"height": 35284},285"id": "DfO6a3SJINnZ",286"outputId": "e5195862-c639-4e03-a754-fee29b56af56"287},288"outputs": [289{290"data": {291"application/vnd.google.colaboratory.intrinsic+json": {292"type": "string"293},294"text/plain": [295"'[MASK]'"296]297},298"execution_count": 4,299"metadata": {},300"output_type": "execute_result"301}302],303"source": [304"tokenizer.mask_token"305]306},307{308"cell_type": "markdown",309"metadata": {310"id": "Xnd_beq0INna"311},312"source": [313"## Masked Language Modeling\n",314"\n",315"Masked Language modelling is a way to perform word prediction that was originally hidden intentionally in a sentence.\n",316"\n",317"In simple terms, it is the task of filling in the blanks.\n",318"\n",319"Masked language modelling can be considered similar to autoencoding modelling which works based on constructing outcomes from unarranged or corrupted input."320]321},322{323"cell_type": "code",324"execution_count": 5,325"metadata": {326"colab": {327"base_uri": "https://localhost:8080/",328"height": 294,329"referenced_widgets": [330"daad56b36f624745a95609ee26a537c1",331"961e2c7863c24f7aacf5a82d917f2d5a",332"538ac7b44e4644cf9702694e2cfd51c9",333"a8e1ba0497a2461782ae307d957b7035",334"0015217dbc9547979e244b044d45e5f4",335"860a7d8016164e9ba0156fa98d6cf269",336"95408d7e0b8044b8b6bfc3a011992c44",337"710533b9a5a1494d95aa7b3e99ca307a",338"ac888a1f4b764839a5e75ec3876d56f5",339"67861cf7f6dc44e59e9850e897905a4b",340"307fdf82a7a34cb891aa5e6221ebc126"341]342},343"id": "FnNpzW3m1cjV",344"outputId": "dbf94d38-5231-4643-e0fe-eca4d1b17e17"345},346"outputs": [347{348"data": {349"application/vnd.jupyter.widget-view+json": {350"model_id": "daad56b36f624745a95609ee26a537c1",351"version_major": 2,352"version_minor": 0353},354"text/plain": [355"Downloading: 0%| | 0.00/420M [00:00<?, ?B/s]"356]357},358"metadata": {},359"output_type": "display_data"360},361{362"name": "stderr",363"output_type": "stream",364"text": [365"Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForMaskedLM: ['cls.seq_relationship.weight', 'cls.seq_relationship.bias']\n",366"- This IS expected if you are initializing BertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",367"- This IS NOT expected if you are initializing BertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"368]369},370{371"name": "stdout",372"output_type": "stream",373"text": [374"The Opera House in Australia is in , sydney city\n",375"The Opera House in Australia is in , melbourne city\n",376"The Opera House in Australia is in , brisbane city\n",377"The Opera House in Australia is in , adelaide city\n",378"The Opera House in Australia is in , the city\n",379"The Opera House in Australia is in , canberra city\n",380"The Opera House in Australia is in , auckland city\n",381"The Opera House in Australia is in , hobart city\n",382"The Opera House in Australia is in , griffith city\n",383"The Opera House in Australia is in , hume city\n"384]385}386],387"source": [388"from transformers import BertTokenizer, BertForMaskedLM\n",389"from torch.nn import functional as F\n",390"import torch\n",391"\n",392"\n",393"tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",394"\n",395"model = BertForMaskedLM.from_pretrained('bert-base-uncased', return_dict = True)\n",396"\n",397"''' Masked Language Modeling works by inserting a mask token at the desired position where you want to predict the best candidate word that would go in that position.\n",398"\n",399"You can simply insert the mask token by concatenating it at the desired position\n",400"\n",401"The Bert Model for Masked Language Modeling predicts the best word/token in its vocabulary that would replace that word. \n",402"\n",403"The logits are the output of the BERT Model before a softmax activation function is applied to the output of BERT. \n",404"i.e. logits are the Prediction scores of the language modeling head (scores for each vocabulary token before SoftMax).\n",405"\n",406"And in order to get the logits, we have to specify \"return_dict = True\" in the parameters when initializing the model, otherwise, the above code will result in a compilation error. \n",407"\n",408"\"return_dict\" - If set to True, the model will return a ModelOutput class instead of a plain tuple.\n",409"\n",410"On \"return_dict = True\" see my note - HuggingFace/input_dict_true_its_purpose.ipynb\n",411"\n",412"'''\n",413"\n",414"text = \"The Opera House in Australia is in , \" + tokenizer.mask_token + \" city\"\n",415"\n",416"input = tokenizer.encode_plus(text, return_tensors = \"pt\")\n",417"\n",418"\n",419"''' In order to get the tensor of softmax values of all the words in BERT’s vocabulary for replacing the mask token, we need to specify the masked token index.\n",420"\n",421"And these we can get using torch.where(). And in this particular example I am retrieving the top 10 candidate replacement words for the mask token. '''\n",422"mask_index = torch.where(input[\"input_ids\"][0] == tokenizer.mask_token_id)\n",423"\n",424"''' mask_token (str or tokenizers.AddedToken, optional) — A special token representing a masked token (used by masked-language modeling pretraining objectives, like BERT). Will be associated to self.mask_token and self.mask_token_id. '''\n",425"\n",426"output = model(**input)\n",427"\n",428"logits = output.logits\n",429"\n",430"''' After we pass the input encoding into the BERT Model, we can get the logits simply by specifying output.logits, which returns a tensor, and after this we can finally apply a softmax activation function to the logits. '''\n",431"\n",432"softmax = F.softmax(logits, dim = -1)\n",433"''' By applying a softmax onto the output of BERT, we get probabilistic distributions for each of the words in BERT’s vocabulary. Word’s with a higher probability value will be better candidate replacement words for the mask token. '''\n",434"\n",435"mask_word = softmax[0, mask_index, :]\n",436"''' In order to get the tensor of softmax values of all the words in BERT’s vocabulary for replacing the mask token, we can specify the masked token index, which we already got using torch.where(). \n",437"\n",438"Further, Because in this particular example I am retrieving the top 10 candidate replacement words for the mask token. '''\n",439"\n",440"top_10 = torch.topk(mask_word, 10, dim = 1)[1][0]\n",441"''' I used the torch.topk() function, which allows you to retrieve the top k values in a given tensor, and it returns a tensor containing those top k values. '''\n",442"\n",443"''' After this, the process becomes relatively simple, as all we have to do is iterate through the tensor, and replace the mask token in the sentence with the candidate token. '''\n",444"for token in top_10:\n",445" word = tokenizer.decode([token])\n",446" new_sentence = text.replace(tokenizer.mask_token, word)\n",447" print(new_sentence)"448]449},450{451"cell_type": "markdown",452"metadata": {453"id": "e1Aq2bb2INnc"454},455"source": [456"If you want to only get the top candidate word, you can do this:\n",457"\n",458"```py\n",459"softmax = F.softmax(logits, dim = -1)\n",460"\n",461"mask_word = softmax[0, mask_index, :]\n",462"\n",463"top_word = torch.argmax(mask_word, dim=1)\n",464"\n",465"print(tokenizer.decode(top_word))\n",466"```\n",467"\n",468"Instead of using torch.topk() for retrieving the top 10 values, we just use torch.argmax(), which returns the index of the maximum value in the tensor. The rest of the code is pretty much the same thing as the original code."469]470},471{472"cell_type": "markdown",473"metadata": {474"id": "J4C51g_XINnd"475},476"source": [477"---\n",478"\n",479"## Next Sentence Prediction\n",480"\n",481"Next Sentence Prediction is the task of predicting whether one sentence follows another sentence. Here is my code for this:\n",482"\n",483"\n",484"[BertForNextSentencePrediction](https://huggingface.co/transformers/v4.9.2/model_doc/bert.html#bertfornextsentenceprediction)\n",485"\n",486"It returns logits (torch.FloatTensor of shape (batch_size, 2)) – Prediction scores of the next sequence prediction (classification) head (scores of True/False continuation before SoftMax)."487]488},489{490"cell_type": "code",491"execution_count": 6,492"metadata": {493"colab": {494"base_uri": "https://localhost:8080/"495},496"id": "dru3GChd1cdI",497"outputId": "7e1608d8-2178-4854-9147-bed300ba1f59"498},499"outputs": [500{501"name": "stderr",502"output_type": "stream",503"text": [504"Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertForNextSentencePrediction: ['cls.predictions.transform.dense.bias', 'cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.decoder.weight', 'cls.predictions.transform.LayerNorm.bias']\n",505"- This IS expected if you are initializing BertForNextSentencePrediction from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",506"- This IS NOT expected if you are initializing BertForNextSentencePrediction from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n"507]508},509{510"name": "stdout",511"output_type": "stream",512"text": [513"tensor([[9.9998e-01, 1.5085e-05]], grad_fn=<SoftmaxBackward0>)\n"514]515}516],517"source": [518"from transformers import BertTokenizer, BertForNextSentencePrediction\n",519"import torch\n",520"from torch.nn import functional as F\n",521"tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')\n",522"model = BertForNextSentencePrediction.from_pretrained('bert-base-uncased')\n",523"\n",524"prompt = \"I came back from Office in the evening\"\n",525"\n",526"next_sentence = \"I opened my Beer after Office\"\n",527"\n",528"encoding = tokenizer.encode_plus(prompt, next_sentence, return_tensors='pt')\n",529"outputs = model(**encoding)[0]\n",530"softmax = F.softmax(outputs, dim = 1)\n",531"print(softmax)"532]533},534{535"cell_type": "markdown",536"metadata": {537"id": "q_rEIxlTINne"538},539"source": [540"Next Sentence prediction is the task of predicting how good a sentence is as a next sentence for a given sentence. \n",541"\n",542"In this case, prompt variable is the given sentence and we are trying to predict whether next_sentence is the next sentence. \n",543"\n",544"\n",545"To do this, the BERT tokenizer automatically inserts a [SEP] token in between the sentences, which represents the separation between the two sentences, and the specific Bert \n",546"\n",547"For Next Sentence Prediction model predicts two values of whether the sentence is the next sentence. \n",548"\n",549"Bert returns two values in a tensor: the first value represents whether the second sentence is a continuation of the first, and the second value represents whether the second sentence is a random sequence or not a good continuation of the first. \n",550"\n",551"### Unlike Masked Language Modeling, we don’t retrieve any logits because we are not trying to compute a softmax on the vocabulary of BERT; \n",552"\n",553"### We are simply trying to compute a softmax on the two values that BERT for next sentence prediction returns so that we can see which value has the highest probability. \n",554"\n",555"### And this will represent whether the second sentence is a good next sentence for the first. \n",556"\n",557"\n",558"Once we get the softmax values, we can simply look at the tensor by printing it out. Here are the values that I got:\n",559"\n",560"`tensor([[0.9953, 0.0047]], grad_fn=<SoftmaxBackward0>)`\n",561"\n",562"Because the first value is considerably higher than the second index, BERT believes that the second sentence follows the first sentence, which is the correct answer."563]564},565{566"cell_type": "markdown",567"metadata": {568"id": "rZVdtgwm35R2"569},570"source": [571"---\n",572"\n",573"## Question Answering"574]575},576{577"cell_type": "code",578"execution_count": null,579"metadata": {},580"outputs": [],581"source": [582"from transformers import BertTokenizer, BertForQuestionAnswering\n",583"import torch\n",584"\n",585"tokenizer = BertTokenizer.from_pretrained(\"deepset/bert-base-cased-squad2\")\n",586"\n",587"model = BertForQuestionAnswering.from_pretrained(\"deepset/bert-base-cased-squad2\")\n",588"\n",589"example_text = \"GPT-3 came in 2020\"\n",590"\n",591"example_question = \"When did GPT-3 come\"\n",592"\n",593"# We can use our tokenizer to automatically generate 2 sentence by passing the\n",594"# two sequences to tokenizer as two arguments\n",595"tokenized_inputs = tokenizer(example_question, example_text, return_tensors=\"pt\")\n",596"tokenized_inputs"597]598},599{600"cell_type": "markdown",601"metadata": {},602"source": [603"\n",604"Bert QA appends question before context.\n",605"\n",606"Tokenizer returns 3 tensors for us.\n",607"\n",608"#### “inputs_ids” are tokenized ids of text.\n",609"\n",610"----------------------\n",611"\n",612"### \"'token_type_ids' => To understand them first note, Some models’ purpose is to do classification on pairs of sentences or question answering.\n",613"\n",614"https://huggingface.co/docs/transformers/v4.20.1/en/glossary#token-type-ids\n",615"\n",616"These require two different sequences to be joined in a single “input_ids” entry, which usually is performed with the help of special tokens, such as the classifier ([CLS]) and separator ([SEP]) tokens. For example, the BERT model builds its two sequence input as such:\n",617"\n",618"\n",619"### [CLS] SEQUENCE_A [SEP] SEQUENCE_B [SEP]\n",620"\n",621"We used our tokenizer to automatically generate such a sentence by passing the two sequences to tokenizer as two arguments\n",622"\n",623"BERT has token type IDs (also called segment IDs). They are represented as a binary mask identifying the two types of sequence in the model.\n",624"\n",625"Here those 2 types of sequences are Questions and the Context.\n",626"\n",627"Token type 0 is for question part and 1 context.\n",628"\n",629"\n",630"**The model will tell you at what start and end position of the input_ids the answer to the question will be located.**\n",631"\n",632"----------------------"633]634},635{636"cell_type": "code",637"execution_count": null,638"metadata": {},639"outputs": [],640"source": [641"text = \"The Vatican Apostolic Library (), more commonly called the Vatican Library or simply the Vat, is the library of the Holy See, located in Vatican City. Formally established in 1475, although it is much older, it is one of the oldest libraries in the world and contains one of the most significant collections of historical texts. It has 75,000 codices from throughout history, as well as 1.1 million printed books, which include some 8,500 incunabula. The Vatican Library is a research library for history, law, philosophy, science and theology. The Vatican Library is open to anyone who can document their qualifications and research needs. Photocopies for private study of pages from books published between 1801 and 1990 can be requested in person or by mail. In March 2014, the Vatican Library began an initial four-year project of digitising its collection of manuscripts, to be made available online. The Vatican Secret Archives were separated from the library at the beginning of the 17th century; they contain another 150,000 items. Scholars have traditionally divided the history of the library into five periods, Pre-Lateran, Lateran, Avignon, Pre-Vatican and Vatican. The Pre-Lateran period, comprising the initial days of the library, dated from the earliest days of the Church. Only a handful of volumes survive from this period, though some are very significant.\"\n",642"\n",643"question = \"When was the Vat formally opened?\""644]645},646{647"cell_type": "code",648"execution_count": null,649"metadata": {650"colab": {651"base_uri": "https://localhost:8080/",652"height": 195,653"referenced_widgets": [654"c7e9324f5d3146a3b3d417f61d16bbb8",655"5dd65f9da5574194a0c8c204576901b4",656"bcb0d5e6845c4c0cb581af288bce7779",657"0c1616cfe601441388222eeaf9d48e86",658"e4ad70b104984d5e9865c242f2b5277c",659"1310bd9c39554d96be1f632371708a3d",660"b7b352542cc546d5a4aade82d6c5e49f",661"3f01a89151f6499fbebac9c050bcece4",662"d6126ce7e3bc406590d7d8e347a52564",663"48104f1435254e0080d2dae289001742",664"24ec404af8aa4fd4a191f8bf1699f559",665"96196c915c5b4a9bad28f75324ce6355",666"ce4170d06ca44d328b21b7330ccb778f",667"2bc92a680a98416fbfeaf049c5d9dd69",668"696f66ced7964725946410701af930c0",669"5d33b562051d46dca504ef4d5dd1ffcf",670"d9b8083459154f3cb302af59de47a2fb",671"3f6feb2ddfaf4f38befad489d310771b",672"70941db0cad948019aff605192bc88cc",673"33e98c2827f64b2ab5aefdb71260498d",674"f9ed2b0a534848dba3082401497f128a",675"76f185b97f804cd49895704e8145c877",676"33c44974d46d47d28dfda8bd94e9451f",677"ec4323b52dc4499ba35d26b5d5bc69da",678"853338dbe0974c43bae38ecd6d7b031f",679"96a0db97f619472ab465b7732c01c552",680"35b4934384af4f37881f2b9a8ead151d",681"53f7796c7a4a4d6c97c8609a6e644877",682"db177021898d45d59923571100e8da5b",683"ead8daeb2a8747489bf785329c837e23",684"3f4ca15e08e1407aab5f6cab9801b621",685"0b8d1ca28d084dacb7d3b422db7c0a5e",686"44a6373b435440e589edad690a9e1824",687"2f3cb019164144e1998745c5f05f99c1",688"22320cfa61d1463cb5214e0a982e8760",689"1630b896c088435daa820cd62f2e9217",690"39662fcaf2d24c2d925d610bf118cc8c",691"68fbf2cd12944aa9aba1e07d698024da",692"67772c7e6899452fbddcd406b24b70d0",693"3c33b86a51314e62b117f4a7fb0b1fbb",694"e4e610b34a374b88a7fc5e82c6e4b256",695"322054a7bf784468a9047d0bd7660271",696"6024b6711c2248db99bdb1c997e24d25",697"7e235bbf5e5a4f5ea1662fb4b2f770d5",698"6deee95bde324836a296f136a1ee6b0e",699"857f239216a04435a6d51819165b9feb",700"41dd33f5f06048beb8086f06705a45b8",701"a64e38625d284737a6c15763125c54f7",702"c17dc7e26205465eae8cb3d9d4070474",703"11097b88dbaa43fe83dc5af1444322ac",704"cad3648ea120426a8a2efe12324b8ed5",705"15e192df0e1c4ec18691b21a064fa334",706"b9bb9484d845460ba7a685186cd664dc",707"6fd158c9e05942919efaae991067386e",708"6b335e7770f340eb9fdd1374a02a05ce"709]710},711"id": "okTIrHVt2BuE",712"outputId": "ea176d18-56bd-44ac-bb22-a37e8335195f"713},714"outputs": [],715"source": [716"tokenizer = BertTokenizer.from_pretrained(\"deepset/bert-base-cased-squad2\")\n",717"\n",718"model = BertForQuestionAnswering.from_pretrained(\"deepset/bert-base-cased-squad2\")\n",719"\n",720"tokenized_inputs = tokenizer(question, text, return_tensors=\"pt\")\n",721"\n",722"with torch.no_grad():\n",723" outputs = model(**tokenized_inputs)\n",724"\n",725"answer_start_index = outputs.start_logits.argmax()\n",726"answer_end_index = outputs.end_logits.argmax()\n",727"\n",728"''' start_logits (torch.FloatTensor of shape (batch_size, sequence_length)) — Span-start scores (before SoftMax).\n",729"\n",730"end_logits (torch.FloatTensor of shape (batch_size, sequence_length)) — Span-end scores (before SoftMax). '''\n",731"\n",732"predict_answer_tokens = tokenized_inputs.input_ids[0, answer_start_index : answer_end_index + 1]\n",733"tokenizer.decode(predict_answer_tokens)"734]735}736],737"metadata": {738"colab": {739"collapsed_sections": [],740"name": "BERT from Hugging Face - Few Baseline Usages.ipynb",741"provenance": []742},743"gpuClass": "standard",744"kernelspec": {745"display_name": "Python 3.8.10 64-bit",746"language": "python",747"name": "python3"748},749"language_info": {750"name": "python",751"version": "3.8.10"752},753"vscode": {754"interpreter": {755"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"756}757},758"widgets": {759"application/vnd.jupyter.widget-state+json": {760"0015217dbc9547979e244b044d45e5f4": {761"model_module": "@jupyter-widgets/base",762"model_module_version": "1.2.0",763"model_name": "LayoutModel",764"state": {765"_model_module": "@jupyter-widgets/base",766"_model_module_version": "1.2.0",767"_model_name": "LayoutModel",768"_view_count": null,769"_view_module": "@jupyter-widgets/base",770"_view_module_version": "1.2.0",771"_view_name": "LayoutView",772"align_content": null,773"align_items": null,774"align_self": null,775"border": null,776"bottom": null,777"display": null,778"flex": null,779"flex_flow": null,780"grid_area": null,781"grid_auto_columns": null,782"grid_auto_flow": null,783"grid_auto_rows": null,784"grid_column": null,785"grid_gap": null,786"grid_row": null,787"grid_template_areas": null,788"grid_template_columns": null,789"grid_template_rows": null,790"height": null,791"justify_content": null,792"justify_items": null,793"left": null,794"margin": null,795"max_height": null,796"max_width": null,797"min_height": null,798"min_width": null,799"object_fit": null,800"object_position": null,801"order": null,802"overflow": null,803"overflow_x": null,804"overflow_y": null,805"padding": null,806"right": null,807"top": null,808"visibility": null,809"width": null810}811},812"017d3ebfbf8a46efa1e956bf8f358636": {813"model_module": "@jupyter-widgets/base",814"model_module_version": "1.2.0",815"model_name": "LayoutModel",816"state": {817"_model_module": "@jupyter-widgets/base",818"_model_module_version": "1.2.0",819"_model_name": "LayoutModel",820"_view_count": null,821"_view_module": "@jupyter-widgets/base",822"_view_module_version": "1.2.0",823"_view_name": "LayoutView",824"align_content": null,825"align_items": null,826"align_self": null,827"border": null,828"bottom": null,829"display": null,830"flex": null,831"flex_flow": null,832"grid_area": null,833"grid_auto_columns": null,834"grid_auto_flow": null,835"grid_auto_rows": null,836"grid_column": null,837"grid_gap": null,838"grid_row": null,839"grid_template_areas": null,840"grid_template_columns": null,841"grid_template_rows": null,842"height": null,843"justify_content": null,844"justify_items": null,845"left": null,846"margin": null,847"max_height": null,848"max_width": null,849"min_height": null,850"min_width": null,851"object_fit": null,852"object_position": null,853"order": null,854"overflow": null,855"overflow_x": null,856"overflow_y": null,857"padding": null,858"right": null,859"top": null,860"visibility": null,861"width": null862}863},864"0472e24144684e1e8cb96e015a74a379": {865"model_module": "@jupyter-widgets/controls",866"model_module_version": "1.5.0",867"model_name": "ProgressStyleModel",868"state": {869"_model_module": "@jupyter-widgets/controls",870"_model_module_version": "1.5.0",871"_model_name": "ProgressStyleModel",872"_view_count": null,873"_view_module": "@jupyter-widgets/base",874"_view_module_version": "1.2.0",875"_view_name": "StyleView",876"bar_color": null,877"description_width": ""878}879},880"0b8d1ca28d084dacb7d3b422db7c0a5e": {881"model_module": "@jupyter-widgets/base",882"model_module_version": "1.2.0",883"model_name": "LayoutModel",884"state": {885"_model_module": "@jupyter-widgets/base",886"_model_module_version": "1.2.0",887"_model_name": "LayoutModel",888"_view_count": null,889"_view_module": "@jupyter-widgets/base",890"_view_module_version": "1.2.0",891"_view_name": "LayoutView",892"align_content": null,893"align_items": null,894"align_self": null,895"border": null,896"bottom": null,897"display": null,898"flex": null,899"flex_flow": null,900"grid_area": null,901"grid_auto_columns": null,902"grid_auto_flow": null,903"grid_auto_rows": null,904"grid_column": null,905"grid_gap": null,906"grid_row": null,907"grid_template_areas": null,908"grid_template_columns": null,909"grid_template_rows": null,910"height": null,911"justify_content": null,912"justify_items": null,913"left": null,914"margin": null,915"max_height": null,916"max_width": null,917"min_height": null,918"min_width": null,919"object_fit": null,920"object_position": null,921"order": null,922"overflow": null,923"overflow_x": null,924"overflow_y": null,925"padding": null,926"right": null,927"top": null,928"visibility": null,929"width": null930}931},932"0c1616cfe601441388222eeaf9d48e86": {933"model_module": "@jupyter-widgets/controls",934"model_module_version": "1.5.0",935"model_name": "HTMLModel",936"state": {937"_dom_classes": [],938"_model_module": "@jupyter-widgets/controls",939"_model_module_version": "1.5.0",940"_model_name": "HTMLModel",941"_view_count": null,942"_view_module": "@jupyter-widgets/controls",943"_view_module_version": "1.5.0",944"_view_name": "HTMLView",945"description": "",946"description_tooltip": null,947"layout": "IPY_MODEL_48104f1435254e0080d2dae289001742",948"placeholder": "",949"style": "IPY_MODEL_24ec404af8aa4fd4a191f8bf1699f559",950"value": " 208k/208k [00:00<00:00, 272kB/s]"951}952},953"11097b88dbaa43fe83dc5af1444322ac": {954"model_module": "@jupyter-widgets/base",955"model_module_version": "1.2.0",956"model_name": "LayoutModel",957"state": {958"_model_module": "@jupyter-widgets/base",959"_model_module_version": "1.2.0",960"_model_name": "LayoutModel",961"_view_count": null,962"_view_module": "@jupyter-widgets/base",963"_view_module_version": "1.2.0",964"_view_name": "LayoutView",965"align_content": null,966"align_items": null,967"align_self": null,968"border": null,969"bottom": null,970"display": null,971"flex": null,972"flex_flow": null,973"grid_area": null,974"grid_auto_columns": null,975"grid_auto_flow": null,976"grid_auto_rows": null,977"grid_column": null,978"grid_gap": null,979"grid_row": null,980"grid_template_areas": null,981"grid_template_columns": null,982"grid_template_rows": null,983"height": null,984"justify_content": null,985"justify_items": null,986"left": null,987"margin": null,988"max_height": null,989"max_width": null,990"min_height": null,991"min_width": null,992"object_fit": null,993"object_position": null,994"order": null,995"overflow": null,996"overflow_x": null,997"overflow_y": null,998"padding": null,999"right": null,1000"top": null,1001"visibility": null,1002"width": null1003}1004},1005"1310bd9c39554d96be1f632371708a3d": {1006"model_module": "@jupyter-widgets/base",1007"model_module_version": "1.2.0",1008"model_name": "LayoutModel",1009"state": {1010"_model_module": "@jupyter-widgets/base",1011"_model_module_version": "1.2.0",1012"_model_name": "LayoutModel",1013"_view_count": null,1014"_view_module": "@jupyter-widgets/base",1015"_view_module_version": "1.2.0",1016"_view_name": "LayoutView",1017"align_content": null,1018"align_items": null,1019"align_self": null,1020"border": null,1021"bottom": null,1022"display": null,1023"flex": null,1024"flex_flow": null,1025"grid_area": null,1026"grid_auto_columns": null,1027"grid_auto_flow": null,1028"grid_auto_rows": null,1029"grid_column": null,1030"grid_gap": null,1031"grid_row": null,1032"grid_template_areas": null,1033"grid_template_columns": null,1034"grid_template_rows": null,1035"height": null,1036"justify_content": null,1037"justify_items": null,1038"left": null,1039"margin": null,1040"max_height": null,1041"max_width": null,1042"min_height": null,1043"min_width": null,1044"object_fit": null,1045"object_position": null,1046"order": null,1047"overflow": null,1048"overflow_x": null,1049"overflow_y": null,1050"padding": null,1051"right": null,1052"top": null,1053"visibility": null,1054"width": null1055}1056},1057"15e192df0e1c4ec18691b21a064fa334": {1058"model_module": "@jupyter-widgets/base",1059"model_module_version": "1.2.0",1060"model_name": "LayoutModel",1061"state": {1062"_model_module": "@jupyter-widgets/base",1063"_model_module_version": "1.2.0",1064"_model_name": "LayoutModel",1065"_view_count": null,1066"_view_module": "@jupyter-widgets/base",1067"_view_module_version": "1.2.0",1068"_view_name": "LayoutView",1069"align_content": null,1070"align_items": null,1071"align_self": null,1072"border": null,1073"bottom": null,1074"display": null,1075"flex": null,1076"flex_flow": null,1077"grid_area": null,1078"grid_auto_columns": null,1079"grid_auto_flow": null,1080"grid_auto_rows": null,1081"grid_column": null,1082"grid_gap": null,1083"grid_row": null,1084"grid_template_areas": null,1085"grid_template_columns": null,1086"grid_template_rows": null,1087"height": null,1088"justify_content": null,1089"justify_items": null,1090"left": null,1091"margin": null,1092"max_height": null,1093"max_width": null,1094"min_height": null,1095"min_width": null,1096"object_fit": null,1097"object_position": null,1098"order": null,1099"overflow": null,1100"overflow_x": null,1101"overflow_y": null,1102"padding": null,1103"right": null,1104"top": null,1105"visibility": null,1106"width": null1107}1108},1109"1630b896c088435daa820cd62f2e9217": {1110"model_module": "@jupyter-widgets/controls",1111"model_module_version": "1.5.0",1112"model_name": "FloatProgressModel",1113"state": {1114"_dom_classes": [],1115"_model_module": "@jupyter-widgets/controls",1116"_model_module_version": "1.5.0",1117"_model_name": "FloatProgressModel",1118"_view_count": null,1119"_view_module": "@jupyter-widgets/controls",1120"_view_module_version": "1.5.0",1121"_view_name": "ProgressView",1122"bar_style": "success",1123"description": "",1124"description_tooltip": null,1125"layout": "IPY_MODEL_e4e610b34a374b88a7fc5e82c6e4b256",1126"max": 508,1127"min": 0,1128"orientation": "horizontal",1129"style": "IPY_MODEL_322054a7bf784468a9047d0bd7660271",1130"value": 5081131}1132},1133"22320cfa61d1463cb5214e0a982e8760": {1134"model_module": "@jupyter-widgets/controls",1135"model_module_version": "1.5.0",1136"model_name": "HTMLModel",1137"state": {1138"_dom_classes": [],1139"_model_module": "@jupyter-widgets/controls",1140"_model_module_version": "1.5.0",1141"_model_name": "HTMLModel",1142"_view_count": null,1143"_view_module": "@jupyter-widgets/controls",1144"_view_module_version": "1.5.0",1145"_view_name": "HTMLView",1146"description": "",1147"description_tooltip": null,1148"layout": "IPY_MODEL_67772c7e6899452fbddcd406b24b70d0",1149"placeholder": "",1150"style": "IPY_MODEL_3c33b86a51314e62b117f4a7fb0b1fbb",1151"value": "Downloading: 100%"1152}1153},1154"24ec404af8aa4fd4a191f8bf1699f559": {1155"model_module": "@jupyter-widgets/controls",1156"model_module_version": "1.5.0",1157"model_name": "DescriptionStyleModel",1158"state": {1159"_model_module": "@jupyter-widgets/controls",1160"_model_module_version": "1.5.0",1161"_model_name": "DescriptionStyleModel",1162"_view_count": null,1163"_view_module": "@jupyter-widgets/base",1164"_view_module_version": "1.2.0",1165"_view_name": "StyleView",1166"description_width": ""1167}1168},1169"2bc92a680a98416fbfeaf049c5d9dd69": {1170"model_module": "@jupyter-widgets/controls",1171"model_module_version": "1.5.0",1172"model_name": "FloatProgressModel",1173"state": {1174"_dom_classes": [],1175"_model_module": "@jupyter-widgets/controls",1176"_model_module_version": "1.5.0",1177"_model_name": "FloatProgressModel",1178"_view_count": null,1179"_view_module": "@jupyter-widgets/controls",1180"_view_module_version": "1.5.0",1181"_view_name": "ProgressView",1182"bar_style": "success",1183"description": "",1184"description_tooltip": null,1185"layout": "IPY_MODEL_70941db0cad948019aff605192bc88cc",1186"max": 112,1187"min": 0,1188"orientation": "horizontal",1189"style": "IPY_MODEL_33e98c2827f64b2ab5aefdb71260498d",1190"value": 1121191}1192},1193"2f3cb019164144e1998745c5f05f99c1": {1194"model_module": "@jupyter-widgets/controls",1195"model_module_version": "1.5.0",1196"model_name": "HBoxModel",1197"state": {1198"_dom_classes": [],1199"_model_module": "@jupyter-widgets/controls",1200"_model_module_version": "1.5.0",1201"_model_name": "HBoxModel",1202"_view_count": null,1203"_view_module": "@jupyter-widgets/controls",1204"_view_module_version": "1.5.0",1205"_view_name": "HBoxView",1206"box_style": "",1207"children": [1208"IPY_MODEL_22320cfa61d1463cb5214e0a982e8760",1209"IPY_MODEL_1630b896c088435daa820cd62f2e9217",1210"IPY_MODEL_39662fcaf2d24c2d925d610bf118cc8c"1211],1212"layout": "IPY_MODEL_68fbf2cd12944aa9aba1e07d698024da"1213}1214},1215"307fdf82a7a34cb891aa5e6221ebc126": {1216"model_module": "@jupyter-widgets/controls",1217"model_module_version": "1.5.0",1218"model_name": "DescriptionStyleModel",1219"state": {1220"_model_module": "@jupyter-widgets/controls",1221"_model_module_version": "1.5.0",1222"_model_name": "DescriptionStyleModel",1223"_view_count": null,1224"_view_module": "@jupyter-widgets/base",1225"_view_module_version": "1.2.0",1226"_view_name": "StyleView",1227"description_width": ""1228}1229},1230"3108d55dcc9b407bac67cb8abcc4a2c0": {1231"model_module": "@jupyter-widgets/base",1232"model_module_version": "1.2.0",1233"model_name": "LayoutModel",1234"state": {1235"_model_module": "@jupyter-widgets/base",1236"_model_module_version": "1.2.0",1237"_model_name": "LayoutModel",1238"_view_count": null,1239"_view_module": "@jupyter-widgets/base",1240"_view_module_version": "1.2.0",1241"_view_name": "LayoutView",1242"align_content": null,1243"align_items": null,1244"align_self": null,1245"border": null,1246"bottom": null,1247"display": null,1248"flex": null,1249"flex_flow": null,1250"grid_area": null,1251"grid_auto_columns": null,1252"grid_auto_flow": null,1253"grid_auto_rows": null,1254"grid_column": null,1255"grid_gap": null,1256"grid_row": null,1257"grid_template_areas": null,1258"grid_template_columns": null,1259"grid_template_rows": null,1260"height": null,1261"justify_content": null,1262"justify_items": null,1263"left": null,1264"margin": null,1265"max_height": null,1266"max_width": null,1267"min_height": null,1268"min_width": null,1269"object_fit": null,1270"object_position": null,1271"order": null,1272"overflow": null,1273"overflow_x": null,1274"overflow_y": null,1275"padding": null,1276"right": null,1277"top": null,1278"visibility": null,1279"width": null1280}1281},1282"322054a7bf784468a9047d0bd7660271": {1283"model_module": "@jupyter-widgets/controls",1284"model_module_version": "1.5.0",1285"model_name": "ProgressStyleModel",1286"state": {1287"_model_module": "@jupyter-widgets/controls",1288"_model_module_version": "1.5.0",1289"_model_name": "ProgressStyleModel",1290"_view_count": null,1291"_view_module": "@jupyter-widgets/base",1292"_view_module_version": "1.2.0",1293"_view_name": "StyleView",1294"bar_color": null,1295"description_width": ""1296}1297},1298"33c44974d46d47d28dfda8bd94e9451f": {1299"model_module": "@jupyter-widgets/controls",1300"model_module_version": "1.5.0",1301"model_name": "HBoxModel",1302"state": {1303"_dom_classes": [],1304"_model_module": "@jupyter-widgets/controls",1305"_model_module_version": "1.5.0",1306"_model_name": "HBoxModel",1307"_view_count": null,1308"_view_module": "@jupyter-widgets/controls",1309"_view_module_version": "1.5.0",1310"_view_name": "HBoxView",1311"box_style": "",1312"children": [1313"IPY_MODEL_ec4323b52dc4499ba35d26b5d5bc69da",1314"IPY_MODEL_853338dbe0974c43bae38ecd6d7b031f",1315"IPY_MODEL_96a0db97f619472ab465b7732c01c552"1316],1317"layout": "IPY_MODEL_35b4934384af4f37881f2b9a8ead151d"1318}1319},1320"33e98c2827f64b2ab5aefdb71260498d": {1321"model_module": "@jupyter-widgets/controls",1322"model_module_version": "1.5.0",1323"model_name": "ProgressStyleModel",1324"state": {1325"_model_module": "@jupyter-widgets/controls",1326"_model_module_version": "1.5.0",1327"_model_name": "ProgressStyleModel",1328"_view_count": null,1329"_view_module": "@jupyter-widgets/base",1330"_view_module_version": "1.2.0",1331"_view_name": "StyleView",1332"bar_color": null,1333"description_width": ""1334}1335},1336"35b4934384af4f37881f2b9a8ead151d": {1337"model_module": "@jupyter-widgets/base",1338"model_module_version": "1.2.0",1339"model_name": "LayoutModel",1340"state": {1341"_model_module": "@jupyter-widgets/base",1342"_model_module_version": "1.2.0",1343"_model_name": "LayoutModel",1344"_view_count": null,1345"_view_module": "@jupyter-widgets/base",1346"_view_module_version": "1.2.0",1347"_view_name": "LayoutView",1348"align_content": null,1349"align_items": null,1350"align_self": null,1351"border": null,1352"bottom": null,1353"display": null,1354"flex": null,1355"flex_flow": null,1356"grid_area": null,1357"grid_auto_columns": null,1358"grid_auto_flow": null,1359"grid_auto_rows": null,1360"grid_column": null,1361"grid_gap": null,1362"grid_row": null,1363"grid_template_areas": null,1364"grid_template_columns": null,1365"grid_template_rows": null,1366"height": null,1367"justify_content": null,1368"justify_items": null,1369"left": null,1370"margin": null,1371"max_height": null,1372"max_width": null,1373"min_height": null,1374"min_width": null,1375"object_fit": null,1376"object_position": null,1377"order": null,1378"overflow": null,1379"overflow_x": null,1380"overflow_y": null,1381"padding": null,1382"right": null,1383"top": null,1384"visibility": null,1385"width": null1386}1387},1388"39662fcaf2d24c2d925d610bf118cc8c": {1389"model_module": "@jupyter-widgets/controls",1390"model_module_version": "1.5.0",1391"model_name": "HTMLModel",1392"state": {1393"_dom_classes": [],1394"_model_module": "@jupyter-widgets/controls",1395"_model_module_version": "1.5.0",1396"_model_name": "HTMLModel",1397"_view_count": null,1398"_view_module": "@jupyter-widgets/controls",1399"_view_module_version": "1.5.0",1400"_view_name": "HTMLView",1401"description": "",1402"description_tooltip": null,1403"layout": "IPY_MODEL_6024b6711c2248db99bdb1c997e24d25",1404"placeholder": "",1405"style": "IPY_MODEL_7e235bbf5e5a4f5ea1662fb4b2f770d5",1406"value": " 508/508 [00:00<00:00, 12.6kB/s]"1407}1408},1409"3c33b86a51314e62b117f4a7fb0b1fbb": {1410"model_module": "@jupyter-widgets/controls",1411"model_module_version": "1.5.0",1412"model_name": "DescriptionStyleModel",1413"state": {1414"_model_module": "@jupyter-widgets/controls",1415"_model_module_version": "1.5.0",1416"_model_name": "DescriptionStyleModel",1417"_view_count": null,1418"_view_module": "@jupyter-widgets/base",1419"_view_module_version": "1.2.0",1420"_view_name": "StyleView",1421"description_width": ""1422}1423},1424"3df2c3cfc54647ac9e206ea8287e486e": {1425"model_module": "@jupyter-widgets/controls",1426"model_module_version": "1.5.0",1427"model_name": "HBoxModel",1428"state": {1429"_dom_classes": [],1430"_model_module": "@jupyter-widgets/controls",1431"_model_module_version": "1.5.0",1432"_model_name": "HBoxModel",1433"_view_count": null,1434"_view_module": "@jupyter-widgets/controls",1435"_view_module_version": "1.5.0",1436"_view_name": "HBoxView",1437"box_style": "",1438"children": [1439"IPY_MODEL_8783e4a95be0490990f30a9a7c750a68",1440"IPY_MODEL_6a39bd415eb346289c51fd067cb18700",1441"IPY_MODEL_e67f6c44a3424205815d92dcebfbd609"1442],1443"layout": "IPY_MODEL_d4bf69ad6ca74580a4b35f115c029295"1444}1445},1446"3f01a89151f6499fbebac9c050bcece4": {1447"model_module": "@jupyter-widgets/base",1448"model_module_version": "1.2.0",1449"model_name": "LayoutModel",1450"state": {1451"_model_module": "@jupyter-widgets/base",1452"_model_module_version": "1.2.0",1453"_model_name": "LayoutModel",1454"_view_count": null,1455"_view_module": "@jupyter-widgets/base",1456"_view_module_version": "1.2.0",1457"_view_name": "LayoutView",1458"align_content": null,1459"align_items": null,1460"align_self": null,1461"border": null,1462"bottom": null,1463"display": null,1464"flex": null,1465"flex_flow": null,1466"grid_area": null,1467"grid_auto_columns": null,1468"grid_auto_flow": null,1469"grid_auto_rows": null,1470"grid_column": null,1471"grid_gap": null,1472"grid_row": null,1473"grid_template_areas": null,1474"grid_template_columns": null,1475"grid_template_rows": null,1476"height": null,1477"justify_content": null,1478"justify_items": null,1479"left": null,1480"margin": null,1481"max_height": null,1482"max_width": null,1483"min_height": null,1484"min_width": null,1485"object_fit": null,1486"object_position": null,1487"order": null,1488"overflow": null,1489"overflow_x": null,1490"overflow_y": null,1491"padding": null,1492"right": null,1493"top": null,1494"visibility": null,1495"width": null1496}1497},1498"3f1df1a788224fc589970e4b70b247fd": {1499"model_module": "@jupyter-widgets/controls",1500"model_module_version": "1.5.0",1501"model_name": "HTMLModel",1502"state": {1503"_dom_classes": [],1504"_model_module": "@jupyter-widgets/controls",1505"_model_module_version": "1.5.0",1506"_model_name": "HTMLModel",1507"_view_count": null,1508"_view_module": "@jupyter-widgets/controls",1509"_view_module_version": "1.5.0",1510"_view_name": "HTMLView",1511"description": "",1512"description_tooltip": null,1513"layout": "IPY_MODEL_9005579c02fd4b00b3c935df9deae17f",1514"placeholder": "",1515"style": "IPY_MODEL_7a54636d8b7f4e59bf12ecdd97a8c89a",1516"value": "Downloading: 100%"1517}1518},1519"3f4ca15e08e1407aab5f6cab9801b621": {1520"model_module": "@jupyter-widgets/controls",1521"model_module_version": "1.5.0",1522"model_name": "ProgressStyleModel",1523"state": {1524"_model_module": "@jupyter-widgets/controls",1525"_model_module_version": "1.5.0",1526"_model_name": "ProgressStyleModel",1527"_view_count": null,1528"_view_module": "@jupyter-widgets/base",1529"_view_module_version": "1.2.0",1530"_view_name": "StyleView",1531"bar_color": null,1532"description_width": ""1533}1534},1535"3f6feb2ddfaf4f38befad489d310771b": {1536"model_module": "@jupyter-widgets/controls",1537"model_module_version": "1.5.0",1538"model_name": "DescriptionStyleModel",1539"state": {1540"_model_module": "@jupyter-widgets/controls",1541"_model_module_version": "1.5.0",1542"_model_name": "DescriptionStyleModel",1543"_view_count": null,1544"_view_module": "@jupyter-widgets/base",1545"_view_module_version": "1.2.0",1546"_view_name": "StyleView",1547"description_width": ""1548}1549},1550"41dd33f5f06048beb8086f06705a45b8": {1551"model_module": "@jupyter-widgets/controls",1552"model_module_version": "1.5.0",1553"model_name": "FloatProgressModel",1554"state": {1555"_dom_classes": [],1556"_model_module": "@jupyter-widgets/controls",1557"_model_module_version": "1.5.0",1558"_model_name": "FloatProgressModel",1559"_view_count": null,1560"_view_module": "@jupyter-widgets/controls",1561"_view_module_version": "1.5.0",1562"_view_name": "ProgressView",1563"bar_style": "success",1564"description": "",1565"description_tooltip": null,1566"layout": "IPY_MODEL_15e192df0e1c4ec18691b21a064fa334",1567"max": 433294681,1568"min": 0,1569"orientation": "horizontal",1570"style": "IPY_MODEL_b9bb9484d845460ba7a685186cd664dc",1571"value": 4332946811572}1573},1574"44a6373b435440e589edad690a9e1824": {1575"model_module": "@jupyter-widgets/controls",1576"model_module_version": "1.5.0",1577"model_name": "DescriptionStyleModel",1578"state": {1579"_model_module": "@jupyter-widgets/controls",1580"_model_module_version": "1.5.0",1581"_model_name": "DescriptionStyleModel",1582"_view_count": null,1583"_view_module": "@jupyter-widgets/base",1584"_view_module_version": "1.2.0",1585"_view_name": "StyleView",1586"description_width": ""1587}1588},1589"48104f1435254e0080d2dae289001742": {1590"model_module": "@jupyter-widgets/base",1591"model_module_version": "1.2.0",1592"model_name": "LayoutModel",1593"state": {1594"_model_module": "@jupyter-widgets/base",1595"_model_module_version": "1.2.0",1596"_model_name": "LayoutModel",1597"_view_count": null,1598"_view_module": "@jupyter-widgets/base",1599"_view_module_version": "1.2.0",1600"_view_name": "LayoutView",1601"align_content": null,1602"align_items": null,1603"align_self": null,1604"border": null,1605"bottom": null,1606"display": null,1607"flex": null,1608"flex_flow": null,1609"grid_area": null,1610"grid_auto_columns": null,1611"grid_auto_flow": null,1612"grid_auto_rows": null,1613"grid_column": null,1614"grid_gap": null,1615"grid_row": null,1616"grid_template_areas": null,1617"grid_template_columns": null,1618"grid_template_rows": null,1619"height": null,1620"justify_content": null,1621"justify_items": null,1622"left": null,1623"margin": null,1624"max_height": null,1625"max_width": null,1626"min_height": null,1627"min_width": null,1628"object_fit": null,1629"object_position": null,1630"order": null,1631"overflow": null,1632"overflow_x": null,1633"overflow_y": null,1634"padding": null,1635"right": null,1636"top": null,1637"visibility": null,1638"width": null1639}1640},1641"4d467784792d4761a5e17fcc7025eafd": {1642"model_module": "@jupyter-widgets/controls",1643"model_module_version": "1.5.0",1644"model_name": "FloatProgressModel",1645"state": {1646"_dom_classes": [],1647"_model_module": "@jupyter-widgets/controls",1648"_model_module_version": "1.5.0",1649"_model_name": "FloatProgressModel",1650"_view_count": null,1651"_view_module": "@jupyter-widgets/controls",1652"_view_module_version": "1.5.0",1653"_view_name": "ProgressView",1654"bar_style": "success",1655"description": "",1656"description_tooltip": null,1657"layout": "IPY_MODEL_3108d55dcc9b407bac67cb8abcc4a2c0",1658"max": 28,1659"min": 0,1660"orientation": "horizontal",1661"style": "IPY_MODEL_0472e24144684e1e8cb96e015a74a379",1662"value": 281663}1664},1665"505b5ad2473c4804b6876233f2357b67": {1666"model_module": "@jupyter-widgets/controls",1667"model_module_version": "1.5.0",1668"model_name": "ProgressStyleModel",1669"state": {1670"_model_module": "@jupyter-widgets/controls",1671"_model_module_version": "1.5.0",1672"_model_name": "ProgressStyleModel",1673"_view_count": null,1674"_view_module": "@jupyter-widgets/base",1675"_view_module_version": "1.2.0",1676"_view_name": "StyleView",1677"bar_color": null,1678"description_width": ""1679}1680},1681"538ac7b44e4644cf9702694e2cfd51c9": {1682"model_module": "@jupyter-widgets/controls",1683"model_module_version": "1.5.0",1684"model_name": "FloatProgressModel",1685"state": {1686"_dom_classes": [],1687"_model_module": "@jupyter-widgets/controls",1688"_model_module_version": "1.5.0",1689"_model_name": "FloatProgressModel",1690"_view_count": null,1691"_view_module": "@jupyter-widgets/controls",1692"_view_module_version": "1.5.0",1693"_view_name": "ProgressView",1694"bar_style": "success",1695"description": "",1696"description_tooltip": null,1697"layout": "IPY_MODEL_710533b9a5a1494d95aa7b3e99ca307a",1698"max": 440473133,1699"min": 0,1700"orientation": "horizontal",1701"style": "IPY_MODEL_ac888a1f4b764839a5e75ec3876d56f5",1702"value": 4404731331703}1704},1705"53f7796c7a4a4d6c97c8609a6e644877": {1706"model_module": "@jupyter-widgets/base",1707"model_module_version": "1.2.0",1708"model_name": "LayoutModel",1709"state": {1710"_model_module": "@jupyter-widgets/base",1711"_model_module_version": "1.2.0",1712"_model_name": "LayoutModel",1713"_view_count": null,1714"_view_module": "@jupyter-widgets/base",1715"_view_module_version": "1.2.0",1716"_view_name": "LayoutView",1717"align_content": null,1718"align_items": null,1719"align_self": null,1720"border": null,1721"bottom": null,1722"display": null,1723"flex": null,1724"flex_flow": null,1725"grid_area": null,1726"grid_auto_columns": null,1727"grid_auto_flow": null,1728"grid_auto_rows": null,1729"grid_column": null,1730"grid_gap": null,1731"grid_row": null,1732"grid_template_areas": null,1733"grid_template_columns": null,1734"grid_template_rows": null,1735"height": null,1736"justify_content": null,1737"justify_items": null,1738"left": null,1739"margin": null,1740"max_height": null,1741"max_width": null,1742"min_height": null,1743"min_width": null,1744"object_fit": null,1745"object_position": null,1746"order": null,1747"overflow": null,1748"overflow_x": null,1749"overflow_y": null,1750"padding": null,1751"right": null,1752"top": null,1753"visibility": null,1754"width": null1755}1756},1757"5439561e53d946ea88df73ea64e30e9d": {1758"model_module": "@jupyter-widgets/controls",1759"model_module_version": "1.5.0",1760"model_name": "HBoxModel",1761"state": {1762"_dom_classes": [],1763"_model_module": "@jupyter-widgets/controls",1764"_model_module_version": "1.5.0",1765"_model_name": "HBoxModel",1766"_view_count": null,1767"_view_module": "@jupyter-widgets/controls",1768"_view_module_version": "1.5.0",1769"_view_name": "HBoxView",1770"box_style": "",1771"children": [1772"IPY_MODEL_ca3ab0fc2b304864bee1ce96be875d6c",1773"IPY_MODEL_4d467784792d4761a5e17fcc7025eafd",1774"IPY_MODEL_7455a72139d8459f8d9571173c20077c"1775],1776"layout": "IPY_MODEL_73a2321c0f274add8e77cad80603580a"1777}1778},1779"5d33b562051d46dca504ef4d5dd1ffcf": {1780"model_module": "@jupyter-widgets/base",1781"model_module_version": "1.2.0",1782"model_name": "LayoutModel",1783"state": {1784"_model_module": "@jupyter-widgets/base",1785"_model_module_version": "1.2.0",1786"_model_name": "LayoutModel",1787"_view_count": null,1788"_view_module": "@jupyter-widgets/base",1789"_view_module_version": "1.2.0",1790"_view_name": "LayoutView",1791"align_content": null,1792"align_items": null,1793"align_self": null,1794"border": null,1795"bottom": null,1796"display": null,1797"flex": null,1798"flex_flow": null,1799"grid_area": null,1800"grid_auto_columns": null,1801"grid_auto_flow": null,1802"grid_auto_rows": null,1803"grid_column": null,1804"grid_gap": null,1805"grid_row": null,1806"grid_template_areas": null,1807"grid_template_columns": null,1808"grid_template_rows": null,1809"height": null,1810"justify_content": null,1811"justify_items": null,1812"left": null,1813"margin": null,1814"max_height": null,1815"max_width": null,1816"min_height": null,1817"min_width": null,1818"object_fit": null,1819"object_position": null,1820"order": null,1821"overflow": null,1822"overflow_x": null,1823"overflow_y": null,1824"padding": null,1825"right": null,1826"top": null,1827"visibility": null,1828"width": null1829}1830},1831"5dd65f9da5574194a0c8c204576901b4": {1832"model_module": "@jupyter-widgets/controls",1833"model_module_version": "1.5.0",1834"model_name": "HTMLModel",1835"state": {1836"_dom_classes": [],1837"_model_module": "@jupyter-widgets/controls",1838"_model_module_version": "1.5.0",1839"_model_name": "HTMLModel",1840"_view_count": null,1841"_view_module": "@jupyter-widgets/controls",1842"_view_module_version": "1.5.0",1843"_view_name": "HTMLView",1844"description": "",1845"description_tooltip": null,1846"layout": "IPY_MODEL_1310bd9c39554d96be1f632371708a3d",1847"placeholder": "",1848"style": "IPY_MODEL_b7b352542cc546d5a4aade82d6c5e49f",1849"value": "Downloading: 100%"1850}1851},1852"6024b6711c2248db99bdb1c997e24d25": {1853"model_module": "@jupyter-widgets/base",1854"model_module_version": "1.2.0",1855"model_name": "LayoutModel",1856"state": {1857"_model_module": "@jupyter-widgets/base",1858"_model_module_version": "1.2.0",1859"_model_name": "LayoutModel",1860"_view_count": null,1861"_view_module": "@jupyter-widgets/base",1862"_view_module_version": "1.2.0",1863"_view_name": "LayoutView",1864"align_content": null,1865"align_items": null,1866"align_self": null,1867"border": null,1868"bottom": null,1869"display": null,1870"flex": null,1871"flex_flow": null,1872"grid_area": null,1873"grid_auto_columns": null,1874"grid_auto_flow": null,1875"grid_auto_rows": null,1876"grid_column": null,1877"grid_gap": null,1878"grid_row": null,1879"grid_template_areas": null,1880"grid_template_columns": null,1881"grid_template_rows": null,1882"height": null,1883"justify_content": null,1884"justify_items": null,1885"left": null,1886"margin": null,1887"max_height": null,1888"max_width": null,1889"min_height": null,1890"min_width": null,1891"object_fit": null,1892"object_position": null,1893"order": null,1894"overflow": null,1895"overflow_x": null,1896"overflow_y": null,1897"padding": null,1898"right": null,1899"top": null,1900"visibility": null,1901"width": null1902}1903},1904"67772c7e6899452fbddcd406b24b70d0": {1905"model_module": "@jupyter-widgets/base",1906"model_module_version": "1.2.0",1907"model_name": "LayoutModel",1908"state": {1909"_model_module": "@jupyter-widgets/base",1910"_model_module_version": "1.2.0",1911"_model_name": "LayoutModel",1912"_view_count": null,1913"_view_module": "@jupyter-widgets/base",1914"_view_module_version": "1.2.0",1915"_view_name": "LayoutView",1916"align_content": null,1917"align_items": null,1918"align_self": null,1919"border": null,1920"bottom": null,1921"display": null,1922"flex": null,1923"flex_flow": null,1924"grid_area": null,1925"grid_auto_columns": null,1926"grid_auto_flow": null,1927"grid_auto_rows": null,1928"grid_column": null,1929"grid_gap": null,1930"grid_row": null,1931"grid_template_areas": null,1932"grid_template_columns": null,1933"grid_template_rows": null,1934"height": null,1935"justify_content": null,1936"justify_items": null,1937"left": null,1938"margin": null,1939"max_height": null,1940"max_width": null,1941"min_height": null,1942"min_width": null,1943"object_fit": null,1944"object_position": null,1945"order": null,1946"overflow": null,1947"overflow_x": null,1948"overflow_y": null,1949"padding": null,1950"right": null,1951"top": null,1952"visibility": null,1953"width": null1954}1955},1956"67861cf7f6dc44e59e9850e897905a4b": {1957"model_module": "@jupyter-widgets/base",1958"model_module_version": "1.2.0",1959"model_name": "LayoutModel",1960"state": {1961"_model_module": "@jupyter-widgets/base",1962"_model_module_version": "1.2.0",1963"_model_name": "LayoutModel",1964"_view_count": null,1965"_view_module": "@jupyter-widgets/base",1966"_view_module_version": "1.2.0",1967"_view_name": "LayoutView",1968"align_content": null,1969"align_items": null,1970"align_self": null,1971"border": null,1972"bottom": null,1973"display": null,1974"flex": null,1975"flex_flow": null,1976"grid_area": null,1977"grid_auto_columns": null,1978"grid_auto_flow": null,1979"grid_auto_rows": null,1980"grid_column": null,1981"grid_gap": null,1982"grid_row": null,1983"grid_template_areas": null,1984"grid_template_columns": null,1985"grid_template_rows": null,1986"height": null,1987"justify_content": null,1988"justify_items": null,1989"left": null,1990"margin": null,1991"max_height": null,1992"max_width": null,1993"min_height": null,1994"min_width": null,1995"object_fit": null,1996"object_position": null,1997"order": null,1998"overflow": null,1999"overflow_x": null,2000"overflow_y": null,2001"padding": null,2002"right": null,2003"top": null,2004"visibility": null,2005"width": null2006}2007},2008"68fbf2cd12944aa9aba1e07d698024da": {2009"model_module": "@jupyter-widgets/base",2010"model_module_version": "1.2.0",2011"model_name": "LayoutModel",2012"state": {2013"_model_module": "@jupyter-widgets/base",2014"_model_module_version": "1.2.0",2015"_model_name": "LayoutModel",2016"_view_count": null,2017"_view_module": "@jupyter-widgets/base",2018"_view_module_version": "1.2.0",2019"_view_name": "LayoutView",2020"align_content": null,2021"align_items": null,2022"align_self": null,2023"border": null,2024"bottom": null,2025"display": null,2026"flex": null,2027"flex_flow": null,2028"grid_area": null,2029"grid_auto_columns": null,2030"grid_auto_flow": null,2031"grid_auto_rows": null,2032"grid_column": null,2033"grid_gap": null,2034"grid_row": null,2035"grid_template_areas": null,2036"grid_template_columns": null,2037"grid_template_rows": null,2038"height": null,2039"justify_content": null,2040"justify_items": null,2041"left": null,2042"margin": null,2043"max_height": null,2044"max_width": null,2045"min_height": null,2046"min_width": null,2047"object_fit": null,2048"object_position": null,2049"order": null,2050"overflow": null,2051"overflow_x": null,2052"overflow_y": null,2053"padding": null,2054"right": null,2055"top": null,2056"visibility": null,2057"width": null2058}2059},2060"696f66ced7964725946410701af930c0": {2061"model_module": "@jupyter-widgets/controls",2062"model_module_version": "1.5.0",2063"model_name": "HTMLModel",2064"state": {2065"_dom_classes": [],2066"_model_module": "@jupyter-widgets/controls",2067"_model_module_version": "1.5.0",2068"_model_name": "HTMLModel",2069"_view_count": null,2070"_view_module": "@jupyter-widgets/controls",2071"_view_module_version": "1.5.0",2072"_view_name": "HTMLView",2073"description": "",2074"description_tooltip": null,2075"layout": "IPY_MODEL_f9ed2b0a534848dba3082401497f128a",2076"placeholder": "",2077"style": "IPY_MODEL_76f185b97f804cd49895704e8145c877",2078"value": " 112/112 [00:00<00:00, 1.91kB/s]"2079}2080},2081"6a39bd415eb346289c51fd067cb18700": {2082"model_module": "@jupyter-widgets/controls",2083"model_module_version": "1.5.0",2084"model_name": "FloatProgressModel",2085"state": {2086"_dom_classes": [],2087"_model_module": "@jupyter-widgets/controls",2088"_model_module_version": "1.5.0",2089"_model_name": "FloatProgressModel",2090"_view_count": null,2091"_view_module": "@jupyter-widgets/controls",2092"_view_module_version": "1.5.0",2093"_view_name": "ProgressView",2094"bar_style": "success",2095"description": "",2096"description_tooltip": null,2097"layout": "IPY_MODEL_ff9c374a59f1400aa0dc26d244ee85dd",2098"max": 231508,2099"min": 0,2100"orientation": "horizontal",2101"style": "IPY_MODEL_505b5ad2473c4804b6876233f2357b67",2102"value": 2315082103}2104},2105"6b335e7770f340eb9fdd1374a02a05ce": {2106"model_module": "@jupyter-widgets/controls",2107"model_module_version": "1.5.0",2108"model_name": "DescriptionStyleModel",2109"state": {2110"_model_module": "@jupyter-widgets/controls",2111"_model_module_version": "1.5.0",2112"_model_name": "DescriptionStyleModel",2113"_view_count": null,2114"_view_module": "@jupyter-widgets/base",2115"_view_module_version": "1.2.0",2116"_view_name": "StyleView",2117"description_width": ""2118}2119},2120"6cc7360fd6e54b29829722243450869a": {2121"model_module": "@jupyter-widgets/base",2122"model_module_version": "1.2.0",2123"model_name": "LayoutModel",2124"state": {2125"_model_module": "@jupyter-widgets/base",2126"_model_module_version": "1.2.0",2127"_model_name": "LayoutModel",2128"_view_count": null,2129"_view_module": "@jupyter-widgets/base",2130"_view_module_version": "1.2.0",2131"_view_name": "LayoutView",2132"align_content": null,2133"align_items": null,2134"align_self": null,2135"border": null,2136"bottom": null,2137"display": null,2138"flex": null,2139"flex_flow": null,2140"grid_area": null,2141"grid_auto_columns": null,2142"grid_auto_flow": null,2143"grid_auto_rows": null,2144"grid_column": null,2145"grid_gap": null,2146"grid_row": null,2147"grid_template_areas": null,2148"grid_template_columns": null,2149"grid_template_rows": null,2150"height": null,2151"justify_content": null,2152"justify_items": null,2153"left": null,2154"margin": null,2155"max_height": null,2156"max_width": null,2157"min_height": null,2158"min_width": null,2159"object_fit": null,2160"object_position": null,2161"order": null,2162"overflow": null,2163"overflow_x": null,2164"overflow_y": null,2165"padding": null,2166"right": null,2167"top": null,2168"visibility": null,2169"width": null2170}2171},2172"6deee95bde324836a296f136a1ee6b0e": {2173"model_module": "@jupyter-widgets/controls",2174"model_module_version": "1.5.0",2175"model_name": "HBoxModel",2176"state": {2177"_dom_classes": [],2178"_model_module": "@jupyter-widgets/controls",2179"_model_module_version": "1.5.0",2180"_model_name": "HBoxModel",2181"_view_count": null,2182"_view_module": "@jupyter-widgets/controls",2183"_view_module_version": "1.5.0",2184"_view_name": "HBoxView",2185"box_style": "",2186"children": [2187"IPY_MODEL_857f239216a04435a6d51819165b9feb",2188"IPY_MODEL_41dd33f5f06048beb8086f06705a45b8",2189"IPY_MODEL_a64e38625d284737a6c15763125c54f7"2190],2191"layout": "IPY_MODEL_c17dc7e26205465eae8cb3d9d4070474"2192}2193},2194"6fd158c9e05942919efaae991067386e": {2195"model_module": "@jupyter-widgets/base",2196"model_module_version": "1.2.0",2197"model_name": "LayoutModel",2198"state": {2199"_model_module": "@jupyter-widgets/base",2200"_model_module_version": "1.2.0",2201"_model_name": "LayoutModel",2202"_view_count": null,2203"_view_module": "@jupyter-widgets/base",2204"_view_module_version": "1.2.0",2205"_view_name": "LayoutView",2206"align_content": null,2207"align_items": null,2208"align_self": null,2209"border": null,2210"bottom": null,2211"display": null,2212"flex": null,2213"flex_flow": null,2214"grid_area": null,2215"grid_auto_columns": null,2216"grid_auto_flow": null,2217"grid_auto_rows": null,2218"grid_column": null,2219"grid_gap": null,2220"grid_row": null,2221"grid_template_areas": null,2222"grid_template_columns": null,2223"grid_template_rows": null,2224"height": null,2225"justify_content": null,2226"justify_items": null,2227"left": null,2228"margin": null,2229"max_height": null,2230"max_width": null,2231"min_height": null,2232"min_width": null,2233"object_fit": null,2234"object_position": null,2235"order": null,2236"overflow": null,2237"overflow_x": null,2238"overflow_y": null,2239"padding": null,2240"right": null,2241"top": null,2242"visibility": null,2243"width": null2244}2245},2246"70941db0cad948019aff605192bc88cc": {2247"model_module": "@jupyter-widgets/base",2248"model_module_version": "1.2.0",2249"model_name": "LayoutModel",2250"state": {2251"_model_module": "@jupyter-widgets/base",2252"_model_module_version": "1.2.0",2253"_model_name": "LayoutModel",2254"_view_count": null,2255"_view_module": "@jupyter-widgets/base",2256"_view_module_version": "1.2.0",2257"_view_name": "LayoutView",2258"align_content": null,2259"align_items": null,2260"align_self": null,2261"border": null,2262"bottom": null,2263"display": null,2264"flex": null,2265"flex_flow": null,2266"grid_area": null,2267"grid_auto_columns": null,2268"grid_auto_flow": null,2269"grid_auto_rows": null,2270"grid_column": null,2271"grid_gap": null,2272"grid_row": null,2273"grid_template_areas": null,2274"grid_template_columns": null,2275"grid_template_rows": null,2276"height": null,2277"justify_content": null,2278"justify_items": null,2279"left": null,2280"margin": null,2281"max_height": null,2282"max_width": null,2283"min_height": null,2284"min_width": null,2285"object_fit": null,2286"object_position": null,2287"order": null,2288"overflow": null,2289"overflow_x": null,2290"overflow_y": null,2291"padding": null,2292"right": null,2293"top": null,2294"visibility": null,2295"width": null2296}2297},2298"710533b9a5a1494d95aa7b3e99ca307a": {2299"model_module": "@jupyter-widgets/base",2300"model_module_version": "1.2.0",2301"model_name": "LayoutModel",2302"state": {2303"_model_module": "@jupyter-widgets/base",2304"_model_module_version": "1.2.0",2305"_model_name": "LayoutModel",2306"_view_count": null,2307"_view_module": "@jupyter-widgets/base",2308"_view_module_version": "1.2.0",2309"_view_name": "LayoutView",2310"align_content": null,2311"align_items": null,2312"align_self": null,2313"border": null,2314"bottom": null,2315"display": null,2316"flex": null,2317"flex_flow": null,2318"grid_area": null,2319"grid_auto_columns": null,2320"grid_auto_flow": null,2321"grid_auto_rows": null,2322"grid_column": null,2323"grid_gap": null,2324"grid_row": null,2325"grid_template_areas": null,2326"grid_template_columns": null,2327"grid_template_rows": null,2328"height": null,2329"justify_content": null,2330"justify_items": null,2331"left": null,2332"margin": null,2333"max_height": null,2334"max_width": null,2335"min_height": null,2336"min_width": null,2337"object_fit": null,2338"object_position": null,2339"order": null,2340"overflow": null,2341"overflow_x": null,2342"overflow_y": null,2343"padding": null,2344"right": null,2345"top": null,2346"visibility": null,2347"width": null2348}2349},2350"73a2321c0f274add8e77cad80603580a": {2351"model_module": "@jupyter-widgets/base",2352"model_module_version": "1.2.0",2353"model_name": "LayoutModel",2354"state": {2355"_model_module": "@jupyter-widgets/base",2356"_model_module_version": "1.2.0",2357"_model_name": "LayoutModel",2358"_view_count": null,2359"_view_module": "@jupyter-widgets/base",2360"_view_module_version": "1.2.0",2361"_view_name": "LayoutView",2362"align_content": null,2363"align_items": null,2364"align_self": null,2365"border": null,2366"bottom": null,2367"display": null,2368"flex": null,2369"flex_flow": null,2370"grid_area": null,2371"grid_auto_columns": null,2372"grid_auto_flow": null,2373"grid_auto_rows": null,2374"grid_column": null,2375"grid_gap": null,2376"grid_row": null,2377"grid_template_areas": null,2378"grid_template_columns": null,2379"grid_template_rows": null,2380"height": null,2381"justify_content": null,2382"justify_items": null,2383"left": null,2384"margin": null,2385"max_height": null,2386"max_width": null,2387"min_height": null,2388"min_width": null,2389"object_fit": null,2390"object_position": null,2391"order": null,2392"overflow": null,2393"overflow_x": null,2394"overflow_y": null,2395"padding": null,2396"right": null,2397"top": null,2398"visibility": null,2399"width": null2400}2401},2402"7455a72139d8459f8d9571173c20077c": {2403"model_module": "@jupyter-widgets/controls",2404"model_module_version": "1.5.0",2405"model_name": "HTMLModel",2406"state": {2407"_dom_classes": [],2408"_model_module": "@jupyter-widgets/controls",2409"_model_module_version": "1.5.0",2410"_model_name": "HTMLModel",2411"_view_count": null,2412"_view_module": "@jupyter-widgets/controls",2413"_view_module_version": "1.5.0",2414"_view_name": "HTMLView",2415"description": "",2416"description_tooltip": null,2417"layout": "IPY_MODEL_b5f908703c444e2d8150adfc755283a7",2418"placeholder": "",2419"style": "IPY_MODEL_aeb4990bc4564386b09ca09bffe423fb",2420"value": " 28.0/28.0 [00:00<00:00, 198B/s]"2421}2422},2423"76f185b97f804cd49895704e8145c877": {2424"model_module": "@jupyter-widgets/controls",2425"model_module_version": "1.5.0",2426"model_name": "DescriptionStyleModel",2427"state": {2428"_model_module": "@jupyter-widgets/controls",2429"_model_module_version": "1.5.0",2430"_model_name": "DescriptionStyleModel",2431"_view_count": null,2432"_view_module": "@jupyter-widgets/base",2433"_view_module_version": "1.2.0",2434"_view_name": "StyleView",2435"description_width": ""2436}2437},2438"7a54636d8b7f4e59bf12ecdd97a8c89a": {2439"model_module": "@jupyter-widgets/controls",2440"model_module_version": "1.5.0",2441"model_name": "DescriptionStyleModel",2442"state": {2443"_model_module": "@jupyter-widgets/controls",2444"_model_module_version": "1.5.0",2445"_model_name": "DescriptionStyleModel",2446"_view_count": null,2447"_view_module": "@jupyter-widgets/base",2448"_view_module_version": "1.2.0",2449"_view_name": "StyleView",2450"description_width": ""2451}2452},2453"7e235bbf5e5a4f5ea1662fb4b2f770d5": {2454"model_module": "@jupyter-widgets/controls",2455"model_module_version": "1.5.0",2456"model_name": "DescriptionStyleModel",2457"state": {2458"_model_module": "@jupyter-widgets/controls",2459"_model_module_version": "1.5.0",2460"_model_name": "DescriptionStyleModel",2461"_view_count": null,2462"_view_module": "@jupyter-widgets/base",2463"_view_module_version": "1.2.0",2464"_view_name": "StyleView",2465"description_width": ""2466}2467},2468"853338dbe0974c43bae38ecd6d7b031f": {2469"model_module": "@jupyter-widgets/controls",2470"model_module_version": "1.5.0",2471"model_name": "FloatProgressModel",2472"state": {2473"_dom_classes": [],2474"_model_module": "@jupyter-widgets/controls",2475"_model_module_version": "1.5.0",2476"_model_name": "FloatProgressModel",2477"_view_count": null,2478"_view_module": "@jupyter-widgets/controls",2479"_view_module_version": "1.5.0",2480"_view_name": "ProgressView",2481"bar_style": "success",2482"description": "",2483"description_tooltip": null,2484"layout": "IPY_MODEL_ead8daeb2a8747489bf785329c837e23",2485"max": 152,2486"min": 0,2487"orientation": "horizontal",2488"style": "IPY_MODEL_3f4ca15e08e1407aab5f6cab9801b621",2489"value": 1522490}2491},2492"857f239216a04435a6d51819165b9feb": {2493"model_module": "@jupyter-widgets/controls",2494"model_module_version": "1.5.0",2495"model_name": "HTMLModel",2496"state": {2497"_dom_classes": [],2498"_model_module": "@jupyter-widgets/controls",2499"_model_module_version": "1.5.0",2500"_model_name": "HTMLModel",2501"_view_count": null,2502"_view_module": "@jupyter-widgets/controls",2503"_view_module_version": "1.5.0",2504"_view_name": "HTMLView",2505"description": "",2506"description_tooltip": null,2507"layout": "IPY_MODEL_11097b88dbaa43fe83dc5af1444322ac",2508"placeholder": "",2509"style": "IPY_MODEL_cad3648ea120426a8a2efe12324b8ed5",2510"value": "Downloading: 100%"2511}2512},2513"860a7d8016164e9ba0156fa98d6cf269": {2514"model_module": "@jupyter-widgets/base",2515"model_module_version": "1.2.0",2516"model_name": "LayoutModel",2517"state": {2518"_model_module": "@jupyter-widgets/base",2519"_model_module_version": "1.2.0",2520"_model_name": "LayoutModel",2521"_view_count": null,2522"_view_module": "@jupyter-widgets/base",2523"_view_module_version": "1.2.0",2524"_view_name": "LayoutView",2525"align_content": null,2526"align_items": null,2527"align_self": null,2528"border": null,2529"bottom": null,2530"display": null,2531"flex": null,2532"flex_flow": null,2533"grid_area": null,2534"grid_auto_columns": null,2535"grid_auto_flow": null,2536"grid_auto_rows": null,2537"grid_column": null,2538"grid_gap": null,2539"grid_row": null,2540"grid_template_areas": null,2541"grid_template_columns": null,2542"grid_template_rows": null,2543"height": null,2544"justify_content": null,2545"justify_items": null,2546"left": null,2547"margin": null,2548"max_height": null,2549"max_width": null,2550"min_height": null,2551"min_width": null,2552"object_fit": null,2553"object_position": null,2554"order": null,2555"overflow": null,2556"overflow_x": null,2557"overflow_y": null,2558"padding": null,2559"right": null,2560"top": null,2561"visibility": null,2562"width": null2563}2564},2565"8783e4a95be0490990f30a9a7c750a68": {2566"model_module": "@jupyter-widgets/controls",2567"model_module_version": "1.5.0",2568"model_name": "HTMLModel",2569"state": {2570"_dom_classes": [],2571"_model_module": "@jupyter-widgets/controls",2572"_model_module_version": "1.5.0",2573"_model_name": "HTMLModel",2574"_view_count": null,2575"_view_module": "@jupyter-widgets/controls",2576"_view_module_version": "1.5.0",2577"_view_name": "HTMLView",2578"description": "",2579"description_tooltip": null,2580"layout": "IPY_MODEL_be6f639e74244154b1e21610f5249fbd",2581"placeholder": "",2582"style": "IPY_MODEL_a129b4bb0cca4aae90a5ff0ee73926e3",2583"value": "Downloading: 100%"2584}2585},2586"8bee921f44d34d709ea0827874e4ac63": {2587"model_module": "@jupyter-widgets/controls",2588"model_module_version": "1.5.0",2589"model_name": "ProgressStyleModel",2590"state": {2591"_model_module": "@jupyter-widgets/controls",2592"_model_module_version": "1.5.0",2593"_model_name": "ProgressStyleModel",2594"_view_count": null,2595"_view_module": "@jupyter-widgets/base",2596"_view_module_version": "1.2.0",2597"_view_name": "StyleView",2598"bar_color": null,2599"description_width": ""2600}2601},2602"9005579c02fd4b00b3c935df9deae17f": {2603"model_module": "@jupyter-widgets/base",2604"model_module_version": "1.2.0",2605"model_name": "LayoutModel",2606"state": {2607"_model_module": "@jupyter-widgets/base",2608"_model_module_version": "1.2.0",2609"_model_name": "LayoutModel",2610"_view_count": null,2611"_view_module": "@jupyter-widgets/base",2612"_view_module_version": "1.2.0",2613"_view_name": "LayoutView",2614"align_content": null,2615"align_items": null,2616"align_self": null,2617"border": null,2618"bottom": null,2619"display": null,2620"flex": null,2621"flex_flow": null,2622"grid_area": null,2623"grid_auto_columns": null,2624"grid_auto_flow": null,2625"grid_auto_rows": null,2626"grid_column": null,2627"grid_gap": null,2628"grid_row": null,2629"grid_template_areas": null,2630"grid_template_columns": null,2631"grid_template_rows": null,2632"height": null,2633"justify_content": null,2634"justify_items": null,2635"left": null,2636"margin": null,2637"max_height": null,2638"max_width": null,2639"min_height": null,2640"min_width": null,2641"object_fit": null,2642"object_position": null,2643"order": null,2644"overflow": null,2645"overflow_x": null,2646"overflow_y": null,2647"padding": null,2648"right": null,2649"top": null,2650"visibility": null,2651"width": null2652}2653},2654"908c9ba59b674a028013c41dcaa9b166": {2655"model_module": "@jupyter-widgets/base",2656"model_module_version": "1.2.0",2657"model_name": "LayoutModel",2658"state": {2659"_model_module": "@jupyter-widgets/base",2660"_model_module_version": "1.2.0",2661"_model_name": "LayoutModel",2662"_view_count": null,2663"_view_module": "@jupyter-widgets/base",2664"_view_module_version": "1.2.0",2665"_view_name": "LayoutView",2666"align_content": null,2667"align_items": null,2668"align_self": null,2669"border": null,2670"bottom": null,2671"display": null,2672"flex": null,2673"flex_flow": null,2674"grid_area": null,2675"grid_auto_columns": null,2676"grid_auto_flow": null,2677"grid_auto_rows": null,2678"grid_column": null,2679"grid_gap": null,2680"grid_row": null,2681"grid_template_areas": null,2682"grid_template_columns": null,2683"grid_template_rows": null,2684"height": null,2685"justify_content": null,2686"justify_items": null,2687"left": null,2688"margin": null,2689"max_height": null,2690"max_width": null,2691"min_height": null,2692"min_width": null,2693"object_fit": null,2694"object_position": null,2695"order": null,2696"overflow": null,2697"overflow_x": null,2698"overflow_y": null,2699"padding": null,2700"right": null,2701"top": null,2702"visibility": null,2703"width": null2704}2705},2706"93b184b6eb584cb98ec731dab7ee81a9": {2707"model_module": "@jupyter-widgets/controls",2708"model_module_version": "1.5.0",2709"model_name": "HBoxModel",2710"state": {2711"_dom_classes": [],2712"_model_module": "@jupyter-widgets/controls",2713"_model_module_version": "1.5.0",2714"_model_name": "HBoxModel",2715"_view_count": null,2716"_view_module": "@jupyter-widgets/controls",2717"_view_module_version": "1.5.0",2718"_view_name": "HBoxView",2719"box_style": "",2720"children": [2721"IPY_MODEL_3f1df1a788224fc589970e4b70b247fd",2722"IPY_MODEL_9537632b2a79434bac79702ea12bff68",2723"IPY_MODEL_bbb1759e97dd4c6c97015014503dc681"2724],2725"layout": "IPY_MODEL_b76f3788d7b5486b8882996076912186"2726}2727},2728"9537632b2a79434bac79702ea12bff68": {2729"model_module": "@jupyter-widgets/controls",2730"model_module_version": "1.5.0",2731"model_name": "FloatProgressModel",2732"state": {2733"_dom_classes": [],2734"_model_module": "@jupyter-widgets/controls",2735"_model_module_version": "1.5.0",2736"_model_name": "FloatProgressModel",2737"_view_count": null,2738"_view_module": "@jupyter-widgets/controls",2739"_view_module_version": "1.5.0",2740"_view_name": "ProgressView",2741"bar_style": "success",2742"description": "",2743"description_tooltip": null,2744"layout": "IPY_MODEL_6cc7360fd6e54b29829722243450869a",2745"max": 570,2746"min": 0,2747"orientation": "horizontal",2748"style": "IPY_MODEL_8bee921f44d34d709ea0827874e4ac63",2749"value": 5702750}2751},2752"95408d7e0b8044b8b6bfc3a011992c44": {2753"model_module": "@jupyter-widgets/controls",2754"model_module_version": "1.5.0",2755"model_name": "DescriptionStyleModel",2756"state": {2757"_model_module": "@jupyter-widgets/controls",2758"_model_module_version": "1.5.0",2759"_model_name": "DescriptionStyleModel",2760"_view_count": null,2761"_view_module": "@jupyter-widgets/base",2762"_view_module_version": "1.2.0",2763"_view_name": "StyleView",2764"description_width": ""2765}2766},2767"96196c915c5b4a9bad28f75324ce6355": {2768"model_module": "@jupyter-widgets/controls",2769"model_module_version": "1.5.0",2770"model_name": "HBoxModel",2771"state": {2772"_dom_classes": [],2773"_model_module": "@jupyter-widgets/controls",2774"_model_module_version": "1.5.0",2775"_model_name": "HBoxModel",2776"_view_count": null,2777"_view_module": "@jupyter-widgets/controls",2778"_view_module_version": "1.5.0",2779"_view_name": "HBoxView",2780"box_style": "",2781"children": [2782"IPY_MODEL_ce4170d06ca44d328b21b7330ccb778f",2783"IPY_MODEL_2bc92a680a98416fbfeaf049c5d9dd69",2784"IPY_MODEL_696f66ced7964725946410701af930c0"2785],2786"layout": "IPY_MODEL_5d33b562051d46dca504ef4d5dd1ffcf"2787}2788},2789"961e2c7863c24f7aacf5a82d917f2d5a": {2790"model_module": "@jupyter-widgets/controls",2791"model_module_version": "1.5.0",2792"model_name": "HTMLModel",2793"state": {2794"_dom_classes": [],2795"_model_module": "@jupyter-widgets/controls",2796"_model_module_version": "1.5.0",2797"_model_name": "HTMLModel",2798"_view_count": null,2799"_view_module": "@jupyter-widgets/controls",2800"_view_module_version": "1.5.0",2801"_view_name": "HTMLView",2802"description": "",2803"description_tooltip": null,2804"layout": "IPY_MODEL_860a7d8016164e9ba0156fa98d6cf269",2805"placeholder": "",2806"style": "IPY_MODEL_95408d7e0b8044b8b6bfc3a011992c44",2807"value": "Downloading: 100%"2808}2809},2810"96a0db97f619472ab465b7732c01c552": {2811"model_module": "@jupyter-widgets/controls",2812"model_module_version": "1.5.0",2813"model_name": "HTMLModel",2814"state": {2815"_dom_classes": [],2816"_model_module": "@jupyter-widgets/controls",2817"_model_module_version": "1.5.0",2818"_model_name": "HTMLModel",2819"_view_count": null,2820"_view_module": "@jupyter-widgets/controls",2821"_view_module_version": "1.5.0",2822"_view_name": "HTMLView",2823"description": "",2824"description_tooltip": null,2825"layout": "IPY_MODEL_0b8d1ca28d084dacb7d3b422db7c0a5e",2826"placeholder": "",2827"style": "IPY_MODEL_44a6373b435440e589edad690a9e1824",2828"value": " 152/152 [00:00<00:00, 4.88kB/s]"2829}2830},2831"96ea79cbc9ec45889d2016d619195168": {2832"model_module": "@jupyter-widgets/controls",2833"model_module_version": "1.5.0",2834"model_name": "DescriptionStyleModel",2835"state": {2836"_model_module": "@jupyter-widgets/controls",2837"_model_module_version": "1.5.0",2838"_model_name": "DescriptionStyleModel",2839"_view_count": null,2840"_view_module": "@jupyter-widgets/base",2841"_view_module_version": "1.2.0",2842"_view_name": "StyleView",2843"description_width": ""2844}2845},2846"a129b4bb0cca4aae90a5ff0ee73926e3": {2847"model_module": "@jupyter-widgets/controls",2848"model_module_version": "1.5.0",2849"model_name": "DescriptionStyleModel",2850"state": {2851"_model_module": "@jupyter-widgets/controls",2852"_model_module_version": "1.5.0",2853"_model_name": "DescriptionStyleModel",2854"_view_count": null,2855"_view_module": "@jupyter-widgets/base",2856"_view_module_version": "1.2.0",2857"_view_name": "StyleView",2858"description_width": ""2859}2860},2861"a64e38625d284737a6c15763125c54f7": {2862"model_module": "@jupyter-widgets/controls",2863"model_module_version": "1.5.0",2864"model_name": "HTMLModel",2865"state": {2866"_dom_classes": [],2867"_model_module": "@jupyter-widgets/controls",2868"_model_module_version": "1.5.0",2869"_model_name": "HTMLModel",2870"_view_count": null,2871"_view_module": "@jupyter-widgets/controls",2872"_view_module_version": "1.5.0",2873"_view_name": "HTMLView",2874"description": "",2875"description_tooltip": null,2876"layout": "IPY_MODEL_6fd158c9e05942919efaae991067386e",2877"placeholder": "",2878"style": "IPY_MODEL_6b335e7770f340eb9fdd1374a02a05ce",2879"value": " 413M/413M [00:15<00:00, 36.8MB/s]"2880}2881},2882"a8e1ba0497a2461782ae307d957b7035": {2883"model_module": "@jupyter-widgets/controls",2884"model_module_version": "1.5.0",2885"model_name": "HTMLModel",2886"state": {2887"_dom_classes": [],2888"_model_module": "@jupyter-widgets/controls",2889"_model_module_version": "1.5.0",2890"_model_name": "HTMLModel",2891"_view_count": null,2892"_view_module": "@jupyter-widgets/controls",2893"_view_module_version": "1.5.0",2894"_view_name": "HTMLView",2895"description": "",2896"description_tooltip": null,2897"layout": "IPY_MODEL_67861cf7f6dc44e59e9850e897905a4b",2898"placeholder": "",2899"style": "IPY_MODEL_307fdf82a7a34cb891aa5e6221ebc126",2900"value": " 420M/420M [00:16<00:00, 39.3MB/s]"2901}2902},2903"a8e763d06bb6456a8d9c62dcfc06111b": {2904"model_module": "@jupyter-widgets/base",2905"model_module_version": "1.2.0",2906"model_name": "LayoutModel",2907"state": {2908"_model_module": "@jupyter-widgets/base",2909"_model_module_version": "1.2.0",2910"_model_name": "LayoutModel",2911"_view_count": null,2912"_view_module": "@jupyter-widgets/base",2913"_view_module_version": "1.2.0",2914"_view_name": "LayoutView",2915"align_content": null,2916"align_items": null,2917"align_self": null,2918"border": null,2919"bottom": null,2920"display": null,2921"flex": null,2922"flex_flow": null,2923"grid_area": null,2924"grid_auto_columns": null,2925"grid_auto_flow": null,2926"grid_auto_rows": null,2927"grid_column": null,2928"grid_gap": null,2929"grid_row": null,2930"grid_template_areas": null,2931"grid_template_columns": null,2932"grid_template_rows": null,2933"height": null,2934"justify_content": null,2935"justify_items": null,2936"left": null,2937"margin": null,2938"max_height": null,2939"max_width": null,2940"min_height": null,2941"min_width": null,2942"object_fit": null,2943"object_position": null,2944"order": null,2945"overflow": null,2946"overflow_x": null,2947"overflow_y": null,2948"padding": null,2949"right": null,2950"top": null,2951"visibility": null,2952"width": null2953}2954},2955"ac888a1f4b764839a5e75ec3876d56f5": {2956"model_module": "@jupyter-widgets/controls",2957"model_module_version": "1.5.0",2958"model_name": "ProgressStyleModel",2959"state": {2960"_model_module": "@jupyter-widgets/controls",2961"_model_module_version": "1.5.0",2962"_model_name": "ProgressStyleModel",2963"_view_count": null,2964"_view_module": "@jupyter-widgets/base",2965"_view_module_version": "1.2.0",2966"_view_name": "StyleView",2967"bar_color": null,2968"description_width": ""2969}2970},2971"aeb4990bc4564386b09ca09bffe423fb": {2972"model_module": "@jupyter-widgets/controls",2973"model_module_version": "1.5.0",2974"model_name": "DescriptionStyleModel",2975"state": {2976"_model_module": "@jupyter-widgets/controls",2977"_model_module_version": "1.5.0",2978"_model_name": "DescriptionStyleModel",2979"_view_count": null,2980"_view_module": "@jupyter-widgets/base",2981"_view_module_version": "1.2.0",2982"_view_name": "StyleView",2983"description_width": ""2984}2985},2986"b1b0b418cabf43309e78cad077e14a72": {2987"model_module": "@jupyter-widgets/controls",2988"model_module_version": "1.5.0",2989"model_name": "DescriptionStyleModel",2990"state": {2991"_model_module": "@jupyter-widgets/controls",2992"_model_module_version": "1.5.0",2993"_model_name": "DescriptionStyleModel",2994"_view_count": null,2995"_view_module": "@jupyter-widgets/base",2996"_view_module_version": "1.2.0",2997"_view_name": "StyleView",2998"description_width": ""2999}3000},3001"b5f908703c444e2d8150adfc755283a7": {3002"model_module": "@jupyter-widgets/base",3003"model_module_version": "1.2.0",3004"model_name": "LayoutModel",3005"state": {3006"_model_module": "@jupyter-widgets/base",3007"_model_module_version": "1.2.0",3008"_model_name": "LayoutModel",3009"_view_count": null,3010"_view_module": "@jupyter-widgets/base",3011"_view_module_version": "1.2.0",3012"_view_name": "LayoutView",3013"align_content": null,3014"align_items": null,3015"align_self": null,3016"border": null,3017"bottom": null,3018"display": null,3019"flex": null,3020"flex_flow": null,3021"grid_area": null,3022"grid_auto_columns": null,3023"grid_auto_flow": null,3024"grid_auto_rows": null,3025"grid_column": null,3026"grid_gap": null,3027"grid_row": null,3028"grid_template_areas": null,3029"grid_template_columns": null,3030"grid_template_rows": null,3031"height": null,3032"justify_content": null,3033"justify_items": null,3034"left": null,3035"margin": null,3036"max_height": null,3037"max_width": null,3038"min_height": null,3039"min_width": null,3040"object_fit": null,3041"object_position": null,3042"order": null,3043"overflow": null,3044"overflow_x": null,3045"overflow_y": null,3046"padding": null,3047"right": null,3048"top": null,3049"visibility": null,3050"width": null3051}3052},3053"b76f3788d7b5486b8882996076912186": {3054"model_module": "@jupyter-widgets/base",3055"model_module_version": "1.2.0",3056"model_name": "LayoutModel",3057"state": {3058"_model_module": "@jupyter-widgets/base",3059"_model_module_version": "1.2.0",3060"_model_name": "LayoutModel",3061"_view_count": null,3062"_view_module": "@jupyter-widgets/base",3063"_view_module_version": "1.2.0",3064"_view_name": "LayoutView",3065"align_content": null,3066"align_items": null,3067"align_self": null,3068"border": null,3069"bottom": null,3070"display": null,3071"flex": null,3072"flex_flow": null,3073"grid_area": null,3074"grid_auto_columns": null,3075"grid_auto_flow": null,3076"grid_auto_rows": null,3077"grid_column": null,3078"grid_gap": null,3079"grid_row": null,3080"grid_template_areas": null,3081"grid_template_columns": null,3082"grid_template_rows": null,3083"height": null,3084"justify_content": null,3085"justify_items": null,3086"left": null,3087"margin": null,3088"max_height": null,3089"max_width": null,3090"min_height": null,3091"min_width": null,3092"object_fit": null,3093"object_position": null,3094"order": null,3095"overflow": null,3096"overflow_x": null,3097"overflow_y": null,3098"padding": null,3099"right": null,3100"top": null,3101"visibility": null,3102"width": null3103}3104},3105"b7b352542cc546d5a4aade82d6c5e49f": {3106"model_module": "@jupyter-widgets/controls",3107"model_module_version": "1.5.0",3108"model_name": "DescriptionStyleModel",3109"state": {3110"_model_module": "@jupyter-widgets/controls",3111"_model_module_version": "1.5.0",3112"_model_name": "DescriptionStyleModel",3113"_view_count": null,3114"_view_module": "@jupyter-widgets/base",3115"_view_module_version": "1.2.0",3116"_view_name": "StyleView",3117"description_width": ""3118}3119},3120"b9bb9484d845460ba7a685186cd664dc": {3121"model_module": "@jupyter-widgets/controls",3122"model_module_version": "1.5.0",3123"model_name": "ProgressStyleModel",3124"state": {3125"_model_module": "@jupyter-widgets/controls",3126"_model_module_version": "1.5.0",3127"_model_name": "ProgressStyleModel",3128"_view_count": null,3129"_view_module": "@jupyter-widgets/base",3130"_view_module_version": "1.2.0",3131"_view_name": "StyleView",3132"bar_color": null,3133"description_width": ""3134}3135},3136"bbb1759e97dd4c6c97015014503dc681": {3137"model_module": "@jupyter-widgets/controls",3138"model_module_version": "1.5.0",3139"model_name": "HTMLModel",3140"state": {3141"_dom_classes": [],3142"_model_module": "@jupyter-widgets/controls",3143"_model_module_version": "1.5.0",3144"_model_name": "HTMLModel",3145"_view_count": null,3146"_view_module": "@jupyter-widgets/controls",3147"_view_module_version": "1.5.0",3148"_view_name": "HTMLView",3149"description": "",3150"description_tooltip": null,3151"layout": "IPY_MODEL_017d3ebfbf8a46efa1e956bf8f358636",3152"placeholder": "",3153"style": "IPY_MODEL_b1b0b418cabf43309e78cad077e14a72",3154"value": " 570/570 [00:00<00:00, 12.4kB/s]"3155}3156},3157"bcb0d5e6845c4c0cb581af288bce7779": {3158"model_module": "@jupyter-widgets/controls",3159"model_module_version": "1.5.0",3160"model_name": "FloatProgressModel",3161"state": {3162"_dom_classes": [],3163"_model_module": "@jupyter-widgets/controls",3164"_model_module_version": "1.5.0",3165"_model_name": "FloatProgressModel",3166"_view_count": null,3167"_view_module": "@jupyter-widgets/controls",3168"_view_module_version": "1.5.0",3169"_view_name": "ProgressView",3170"bar_style": "success",3171"description": "",3172"description_tooltip": null,3173"layout": "IPY_MODEL_3f01a89151f6499fbebac9c050bcece4",3174"max": 213450,3175"min": 0,3176"orientation": "horizontal",3177"style": "IPY_MODEL_d6126ce7e3bc406590d7d8e347a52564",3178"value": 2134503179}3180},3181"be6f639e74244154b1e21610f5249fbd": {3182"model_module": "@jupyter-widgets/base",3183"model_module_version": "1.2.0",3184"model_name": "LayoutModel",3185"state": {3186"_model_module": "@jupyter-widgets/base",3187"_model_module_version": "1.2.0",3188"_model_name": "LayoutModel",3189"_view_count": null,3190"_view_module": "@jupyter-widgets/base",3191"_view_module_version": "1.2.0",3192"_view_name": "LayoutView",3193"align_content": null,3194"align_items": null,3195"align_self": null,3196"border": null,3197"bottom": null,3198"display": null,3199"flex": null,3200"flex_flow": null,3201"grid_area": null,3202"grid_auto_columns": null,3203"grid_auto_flow": null,3204"grid_auto_rows": null,3205"grid_column": null,3206"grid_gap": null,3207"grid_row": null,3208"grid_template_areas": null,3209"grid_template_columns": null,3210"grid_template_rows": null,3211"height": null,3212"justify_content": null,3213"justify_items": null,3214"left": null,3215"margin": null,3216"max_height": null,3217"max_width": null,3218"min_height": null,3219"min_width": null,3220"object_fit": null,3221"object_position": null,3222"order": null,3223"overflow": null,3224"overflow_x": null,3225"overflow_y": null,3226"padding": null,3227"right": null,3228"top": null,3229"visibility": null,3230"width": null3231}3232},3233"c17dc7e26205465eae8cb3d9d4070474": {3234"model_module": "@jupyter-widgets/base",3235"model_module_version": "1.2.0",3236"model_name": "LayoutModel",3237"state": {3238"_model_module": "@jupyter-widgets/base",3239"_model_module_version": "1.2.0",3240"_model_name": "LayoutModel",3241"_view_count": null,3242"_view_module": "@jupyter-widgets/base",3243"_view_module_version": "1.2.0",3244"_view_name": "LayoutView",3245"align_content": null,3246"align_items": null,3247"align_self": null,3248"border": null,3249"bottom": null,3250"display": null,3251"flex": null,3252"flex_flow": null,3253"grid_area": null,3254"grid_auto_columns": null,3255"grid_auto_flow": null,3256"grid_auto_rows": null,3257"grid_column": null,3258"grid_gap": null,3259"grid_row": null,3260"grid_template_areas": null,3261"grid_template_columns": null,3262"grid_template_rows": null,3263"height": null,3264"justify_content": null,3265"justify_items": null,3266"left": null,3267"margin": null,3268"max_height": null,3269"max_width": null,3270"min_height": null,3271"min_width": null,3272"object_fit": null,3273"object_position": null,3274"order": null,3275"overflow": null,3276"overflow_x": null,3277"overflow_y": null,3278"padding": null,3279"right": null,3280"top": null,3281"visibility": null,3282"width": null3283}3284},3285"c70efd624f97461086e67dad24397742": {3286"model_module": "@jupyter-widgets/controls",3287"model_module_version": "1.5.0",3288"model_name": "DescriptionStyleModel",3289"state": {3290"_model_module": "@jupyter-widgets/controls",3291"_model_module_version": "1.5.0",3292"_model_name": "DescriptionStyleModel",3293"_view_count": null,3294"_view_module": "@jupyter-widgets/base",3295"_view_module_version": "1.2.0",3296"_view_name": "StyleView",3297"description_width": ""3298}3299},3300"c7e9324f5d3146a3b3d417f61d16bbb8": {3301"model_module": "@jupyter-widgets/controls",3302"model_module_version": "1.5.0",3303"model_name": "HBoxModel",3304"state": {3305"_dom_classes": [],3306"_model_module": "@jupyter-widgets/controls",3307"_model_module_version": "1.5.0",3308"_model_name": "HBoxModel",3309"_view_count": null,3310"_view_module": "@jupyter-widgets/controls",3311"_view_module_version": "1.5.0",3312"_view_name": "HBoxView",3313"box_style": "",3314"children": [3315"IPY_MODEL_5dd65f9da5574194a0c8c204576901b4",3316"IPY_MODEL_bcb0d5e6845c4c0cb581af288bce7779",3317"IPY_MODEL_0c1616cfe601441388222eeaf9d48e86"3318],3319"layout": "IPY_MODEL_e4ad70b104984d5e9865c242f2b5277c"3320}3321},3322"ca3ab0fc2b304864bee1ce96be875d6c": {3323"model_module": "@jupyter-widgets/controls",3324"model_module_version": "1.5.0",3325"model_name": "HTMLModel",3326"state": {3327"_dom_classes": [],3328"_model_module": "@jupyter-widgets/controls",3329"_model_module_version": "1.5.0",3330"_model_name": "HTMLModel",3331"_view_count": null,3332"_view_module": "@jupyter-widgets/controls",3333"_view_module_version": "1.5.0",3334"_view_name": "HTMLView",3335"description": "",3336"description_tooltip": null,3337"layout": "IPY_MODEL_a8e763d06bb6456a8d9c62dcfc06111b",3338"placeholder": "",3339"style": "IPY_MODEL_c70efd624f97461086e67dad24397742",3340"value": "Downloading: 100%"3341}3342},3343"cad3648ea120426a8a2efe12324b8ed5": {3344"model_module": "@jupyter-widgets/controls",3345"model_module_version": "1.5.0",3346"model_name": "DescriptionStyleModel",3347"state": {3348"_model_module": "@jupyter-widgets/controls",3349"_model_module_version": "1.5.0",3350"_model_name": "DescriptionStyleModel",3351"_view_count": null,3352"_view_module": "@jupyter-widgets/base",3353"_view_module_version": "1.2.0",3354"_view_name": "StyleView",3355"description_width": ""3356}3357},3358"ce4170d06ca44d328b21b7330ccb778f": {3359"model_module": "@jupyter-widgets/controls",3360"model_module_version": "1.5.0",3361"model_name": "HTMLModel",3362"state": {3363"_dom_classes": [],3364"_model_module": "@jupyter-widgets/controls",3365"_model_module_version": "1.5.0",3366"_model_name": "HTMLModel",3367"_view_count": null,3368"_view_module": "@jupyter-widgets/controls",3369"_view_module_version": "1.5.0",3370"_view_name": "HTMLView",3371"description": "",3372"description_tooltip": null,3373"layout": "IPY_MODEL_d9b8083459154f3cb302af59de47a2fb",3374"placeholder": "",3375"style": "IPY_MODEL_3f6feb2ddfaf4f38befad489d310771b",3376"value": "Downloading: 100%"3377}3378},3379"d4bf69ad6ca74580a4b35f115c029295": {3380"model_module": "@jupyter-widgets/base",3381"model_module_version": "1.2.0",3382"model_name": "LayoutModel",3383"state": {3384"_model_module": "@jupyter-widgets/base",3385"_model_module_version": "1.2.0",3386"_model_name": "LayoutModel",3387"_view_count": null,3388"_view_module": "@jupyter-widgets/base",3389"_view_module_version": "1.2.0",3390"_view_name": "LayoutView",3391"align_content": null,3392"align_items": null,3393"align_self": null,3394"border": null,3395"bottom": null,3396"display": null,3397"flex": null,3398"flex_flow": null,3399"grid_area": null,3400"grid_auto_columns": null,3401"grid_auto_flow": null,3402"grid_auto_rows": null,3403"grid_column": null,3404"grid_gap": null,3405"grid_row": null,3406"grid_template_areas": null,3407"grid_template_columns": null,3408"grid_template_rows": null,3409"height": null,3410"justify_content": null,3411"justify_items": null,3412"left": null,3413"margin": null,3414"max_height": null,3415"max_width": null,3416"min_height": null,3417"min_width": null,3418"object_fit": null,3419"object_position": null,3420"order": null,3421"overflow": null,3422"overflow_x": null,3423"overflow_y": null,3424"padding": null,3425"right": null,3426"top": null,3427"visibility": null,3428"width": null3429}3430},3431"d6126ce7e3bc406590d7d8e347a52564": {3432"model_module": "@jupyter-widgets/controls",3433"model_module_version": "1.5.0",3434"model_name": "ProgressStyleModel",3435"state": {3436"_model_module": "@jupyter-widgets/controls",3437"_model_module_version": "1.5.0",3438"_model_name": "ProgressStyleModel",3439"_view_count": null,3440"_view_module": "@jupyter-widgets/base",3441"_view_module_version": "1.2.0",3442"_view_name": "StyleView",3443"bar_color": null,3444"description_width": ""3445}3446},3447"d9b8083459154f3cb302af59de47a2fb": {3448"model_module": "@jupyter-widgets/base",3449"model_module_version": "1.2.0",3450"model_name": "LayoutModel",3451"state": {3452"_model_module": "@jupyter-widgets/base",3453"_model_module_version": "1.2.0",3454"_model_name": "LayoutModel",3455"_view_count": null,3456"_view_module": "@jupyter-widgets/base",3457"_view_module_version": "1.2.0",3458"_view_name": "LayoutView",3459"align_content": null,3460"align_items": null,3461"align_self": null,3462"border": null,3463"bottom": null,3464"display": null,3465"flex": null,3466"flex_flow": null,3467"grid_area": null,3468"grid_auto_columns": null,3469"grid_auto_flow": null,3470"grid_auto_rows": null,3471"grid_column": null,3472"grid_gap": null,3473"grid_row": null,3474"grid_template_areas": null,3475"grid_template_columns": null,3476"grid_template_rows": null,3477"height": null,3478"justify_content": null,3479"justify_items": null,3480"left": null,3481"margin": null,3482"max_height": null,3483"max_width": null,3484"min_height": null,3485"min_width": null,3486"object_fit": null,3487"object_position": null,3488"order": null,3489"overflow": null,3490"overflow_x": null,3491"overflow_y": null,3492"padding": null,3493"right": null,3494"top": null,3495"visibility": null,3496"width": null3497}3498},3499"daad56b36f624745a95609ee26a537c1": {3500"model_module": "@jupyter-widgets/controls",3501"model_module_version": "1.5.0",3502"model_name": "HBoxModel",3503"state": {3504"_dom_classes": [],3505"_model_module": "@jupyter-widgets/controls",3506"_model_module_version": "1.5.0",3507"_model_name": "HBoxModel",3508"_view_count": null,3509"_view_module": "@jupyter-widgets/controls",3510"_view_module_version": "1.5.0",3511"_view_name": "HBoxView",3512"box_style": "",3513"children": [3514"IPY_MODEL_961e2c7863c24f7aacf5a82d917f2d5a",3515"IPY_MODEL_538ac7b44e4644cf9702694e2cfd51c9",3516"IPY_MODEL_a8e1ba0497a2461782ae307d957b7035"3517],3518"layout": "IPY_MODEL_0015217dbc9547979e244b044d45e5f4"3519}3520},3521"db177021898d45d59923571100e8da5b": {3522"model_module": "@jupyter-widgets/controls",3523"model_module_version": "1.5.0",3524"model_name": "DescriptionStyleModel",3525"state": {3526"_model_module": "@jupyter-widgets/controls",3527"_model_module_version": "1.5.0",3528"_model_name": "DescriptionStyleModel",3529"_view_count": null,3530"_view_module": "@jupyter-widgets/base",3531"_view_module_version": "1.2.0",3532"_view_name": "StyleView",3533"description_width": ""3534}3535},3536"e4ad70b104984d5e9865c242f2b5277c": {3537"model_module": "@jupyter-widgets/base",3538"model_module_version": "1.2.0",3539"model_name": "LayoutModel",3540"state": {3541"_model_module": "@jupyter-widgets/base",3542"_model_module_version": "1.2.0",3543"_model_name": "LayoutModel",3544"_view_count": null,3545"_view_module": "@jupyter-widgets/base",3546"_view_module_version": "1.2.0",3547"_view_name": "LayoutView",3548"align_content": null,3549"align_items": null,3550"align_self": null,3551"border": null,3552"bottom": null,3553"display": null,3554"flex": null,3555"flex_flow": null,3556"grid_area": null,3557"grid_auto_columns": null,3558"grid_auto_flow": null,3559"grid_auto_rows": null,3560"grid_column": null,3561"grid_gap": null,3562"grid_row": null,3563"grid_template_areas": null,3564"grid_template_columns": null,3565"grid_template_rows": null,3566"height": null,3567"justify_content": null,3568"justify_items": null,3569"left": null,3570"margin": null,3571"max_height": null,3572"max_width": null,3573"min_height": null,3574"min_width": null,3575"object_fit": null,3576"object_position": null,3577"order": null,3578"overflow": null,3579"overflow_x": null,3580"overflow_y": null,3581"padding": null,3582"right": null,3583"top": null,3584"visibility": null,3585"width": null3586}3587},3588"e4e610b34a374b88a7fc5e82c6e4b256": {3589"model_module": "@jupyter-widgets/base",3590"model_module_version": "1.2.0",3591"model_name": "LayoutModel",3592"state": {3593"_model_module": "@jupyter-widgets/base",3594"_model_module_version": "1.2.0",3595"_model_name": "LayoutModel",3596"_view_count": null,3597"_view_module": "@jupyter-widgets/base",3598"_view_module_version": "1.2.0",3599"_view_name": "LayoutView",3600"align_content": null,3601"align_items": null,3602"align_self": null,3603"border": null,3604"bottom": null,3605"display": null,3606"flex": null,3607"flex_flow": null,3608"grid_area": null,3609"grid_auto_columns": null,3610"grid_auto_flow": null,3611"grid_auto_rows": null,3612"grid_column": null,3613"grid_gap": null,3614"grid_row": null,3615"grid_template_areas": null,3616"grid_template_columns": null,3617"grid_template_rows": null,3618"height": null,3619"justify_content": null,3620"justify_items": null,3621"left": null,3622"margin": null,3623"max_height": null,3624"max_width": null,3625"min_height": null,3626"min_width": null,3627"object_fit": null,3628"object_position": null,3629"order": null,3630"overflow": null,3631"overflow_x": null,3632"overflow_y": null,3633"padding": null,3634"right": null,3635"top": null,3636"visibility": null,3637"width": null3638}3639},3640"e67f6c44a3424205815d92dcebfbd609": {3641"model_module": "@jupyter-widgets/controls",3642"model_module_version": "1.5.0",3643"model_name": "HTMLModel",3644"state": {3645"_dom_classes": [],3646"_model_module": "@jupyter-widgets/controls",3647"_model_module_version": "1.5.0",3648"_model_name": "HTMLModel",3649"_view_count": null,3650"_view_module": "@jupyter-widgets/controls",3651"_view_module_version": "1.5.0",3652"_view_name": "HTMLView",3653"description": "",3654"description_tooltip": null,3655"layout": "IPY_MODEL_908c9ba59b674a028013c41dcaa9b166",3656"placeholder": "",3657"style": "IPY_MODEL_96ea79cbc9ec45889d2016d619195168",3658"value": " 226k/226k [00:00<00:00, 309kB/s]"3659}3660},3661"ead8daeb2a8747489bf785329c837e23": {3662"model_module": "@jupyter-widgets/base",3663"model_module_version": "1.2.0",3664"model_name": "LayoutModel",3665"state": {3666"_model_module": "@jupyter-widgets/base",3667"_model_module_version": "1.2.0",3668"_model_name": "LayoutModel",3669"_view_count": null,3670"_view_module": "@jupyter-widgets/base",3671"_view_module_version": "1.2.0",3672"_view_name": "LayoutView",3673"align_content": null,3674"align_items": null,3675"align_self": null,3676"border": null,3677"bottom": null,3678"display": null,3679"flex": null,3680"flex_flow": null,3681"grid_area": null,3682"grid_auto_columns": null,3683"grid_auto_flow": null,3684"grid_auto_rows": null,3685"grid_column": null,3686"grid_gap": null,3687"grid_row": null,3688"grid_template_areas": null,3689"grid_template_columns": null,3690"grid_template_rows": null,3691"height": null,3692"justify_content": null,3693"justify_items": null,3694"left": null,3695"margin": null,3696"max_height": null,3697"max_width": null,3698"min_height": null,3699"min_width": null,3700"object_fit": null,3701"object_position": null,3702"order": null,3703"overflow": null,3704"overflow_x": null,3705"overflow_y": null,3706"padding": null,3707"right": null,3708"top": null,3709"visibility": null,3710"width": null3711}3712},3713"ec4323b52dc4499ba35d26b5d5bc69da": {3714"model_module": "@jupyter-widgets/controls",3715"model_module_version": "1.5.0",3716"model_name": "HTMLModel",3717"state": {3718"_dom_classes": [],3719"_model_module": "@jupyter-widgets/controls",3720"_model_module_version": "1.5.0",3721"_model_name": "HTMLModel",3722"_view_count": null,3723"_view_module": "@jupyter-widgets/controls",3724"_view_module_version": "1.5.0",3725"_view_name": "HTMLView",3726"description": "",3727"description_tooltip": null,3728"layout": "IPY_MODEL_53f7796c7a4a4d6c97c8609a6e644877",3729"placeholder": "",3730"style": "IPY_MODEL_db177021898d45d59923571100e8da5b",3731"value": "Downloading: 100%"3732}3733},3734"f9ed2b0a534848dba3082401497f128a": {3735"model_module": "@jupyter-widgets/base",3736"model_module_version": "1.2.0",3737"model_name": "LayoutModel",3738"state": {3739"_model_module": "@jupyter-widgets/base",3740"_model_module_version": "1.2.0",3741"_model_name": "LayoutModel",3742"_view_count": null,3743"_view_module": "@jupyter-widgets/base",3744"_view_module_version": "1.2.0",3745"_view_name": "LayoutView",3746"align_content": null,3747"align_items": null,3748"align_self": null,3749"border": null,3750"bottom": null,3751"display": null,3752"flex": null,3753"flex_flow": null,3754"grid_area": null,3755"grid_auto_columns": null,3756"grid_auto_flow": null,3757"grid_auto_rows": null,3758"grid_column": null,3759"grid_gap": null,3760"grid_row": null,3761"grid_template_areas": null,3762"grid_template_columns": null,3763"grid_template_rows": null,3764"height": null,3765"justify_content": null,3766"justify_items": null,3767"left": null,3768"margin": null,3769"max_height": null,3770"max_width": null,3771"min_height": null,3772"min_width": null,3773"object_fit": null,3774"object_position": null,3775"order": null,3776"overflow": null,3777"overflow_x": null,3778"overflow_y": null,3779"padding": null,3780"right": null,3781"top": null,3782"visibility": null,3783"width": null3784}3785},3786"ff9c374a59f1400aa0dc26d244ee85dd": {3787"model_module": "@jupyter-widgets/base",3788"model_module_version": "1.2.0",3789"model_name": "LayoutModel",3790"state": {3791"_model_module": "@jupyter-widgets/base",3792"_model_module_version": "1.2.0",3793"_model_name": "LayoutModel",3794"_view_count": null,3795"_view_module": "@jupyter-widgets/base",3796"_view_module_version": "1.2.0",3797"_view_name": "LayoutView",3798"align_content": null,3799"align_items": null,3800"align_self": null,3801"border": null,3802"bottom": null,3803"display": null,3804"flex": null,3805"flex_flow": null,3806"grid_area": null,3807"grid_auto_columns": null,3808"grid_auto_flow": null,3809"grid_auto_rows": null,3810"grid_column": null,3811"grid_gap": null,3812"grid_row": null,3813"grid_template_areas": null,3814"grid_template_columns": null,3815"grid_template_rows": null,3816"height": null,3817"justify_content": null,3818"justify_items": null,3819"left": null,3820"margin": null,3821"max_height": null,3822"max_width": null,3823"min_height": null,3824"min_width": null,3825"object_fit": null,3826"object_position": null,3827"order": null,3828"overflow": null,3829"overflow_x": null,3830"overflow_y": null,3831"padding": null,3832"right": null,3833"top": null,3834"visibility": null,3835"width": null3836}3837}3838}3839}3840},3841"nbformat": 4,3842"nbformat_minor": 03843}3844