examples
898 строк · 45.0 Кб
1{2"cells": [3{4"attachments": {},5"cell_type": "markdown",6"id": "ac993f9f",7"metadata": {},8"source": [9"[](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/analytics-and-ml/model-training/training-with-wandb/03-query.ipynb) [](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/analytics-and-ml/model-training/training-with-wandb/03-query.ipynb)"10]11},12{13"cell_type": "code",14"execution_count": null,15"id": "7f3cf55f",16"metadata": {},17"outputs": [],18"source": [19"!pip install -qq wandb datasets pinecone-client sentence-transformers transformers"20]21},22{23"attachments": {},24"cell_type": "markdown",25"id": "afcfd398",26"metadata": {},27"source": [28"## Encoder Training\n",29"\n",30"This is part *four* of a four-part notebook series on fine-tuning encoder models with Weights & Biases for use with Pinecone. Find the [full set of notebooks on Github here](https://github.com/pinecone-io/examples/blob/master/analytics-and-ml/model-training/training-with-wandb).\n",31"\n",32"We start by initializing our connection to Pinecone. You should already have an [API key from here](https://app.pinecone.io) if following the previous notebooks."33]34},35{36"cell_type": "code",37"execution_count": 1,38"id": "011cbb74-0f0a-46e2-b89a-69bb34cc2f30",39"metadata": {},40"outputs": [],41"source": [42"from pinecone import Pinecone\n",43"\n",44"pinecone.init(\n",45" api_key='YOUR_API_KEY', # app.pinecone.io\n",46" environment='YOUR_ENV' # find next to api key in console\n",47")\n",48"\n",49"index_id = 'arxiv-search'\n",50"\n",51"# connect to index\n",52"index = pinecone.Index(index_id)"53]54},55{56"cell_type": "markdown",57"id": "c57c7158-4b44-495e-912c-5dd3c14ad8f8",58"metadata": {},59"source": [60"We load the model from W&B as before..."61]62},63{64"cell_type": "code",65"execution_count": 2,66"id": "2c443779-13b2-40a0-89da-1e89ada0ad8d",67"metadata": {},68"outputs": [69{70"name": "stderr",71"output_type": "stream",72"text": [73"/opt/conda/lib/python3.7/site-packages/IPython/html.py:14: ShimWarning: The `IPython.html` package has been deprecated since IPython 4.0. You should import from `notebook` instead. `IPython.html.widgets` has moved to `ipywidgets`.\n",74" \"`IPython.html.widgets` has moved to `ipywidgets`.\", ShimWarning)\n",75"\u001b[34m\u001b[1mwandb\u001b[0m: Currently logged in as: \u001b[33mjamesbriggs\u001b[0m. Use \u001b[1m`wandb login --relogin`\u001b[0m to force relogin\n"76]77},78{79"data": {80"text/html": [81"Tracking run with wandb version 0.13.5"82],83"text/plain": [84"<IPython.core.display.HTML object>"85]86},87"metadata": {},88"output_type": "display_data"89},90{91"data": {92"text/html": [93"Run data is saved locally in <code>/home/jupyter/wandb/wandb/run-20221110_083930-2d602b1c</code>"94],95"text/plain": [96"<IPython.core.display.HTML object>"97]98},99"metadata": {},100"output_type": "display_data"101},102{103"data": {104"text/html": [105"Syncing run <strong><a href=\"https://wandb.ai/jamesbriggs/arxiv-searching/runs/2d602b1c\" target=\"_blank\">atomic-oath-36</a></strong> to <a href=\"https://wandb.ai/jamesbriggs/arxiv-searching\" target=\"_blank\">Weights & Biases</a> (<a href=\"https://wandb.me/run\" target=\"_blank\">docs</a>)<br/>"106],107"text/plain": [108"<IPython.core.display.HTML object>"109]110},111"metadata": {},112"output_type": "display_data"113},114{115"name": "stderr",116"output_type": "stream",117"text": [118"\u001b[34m\u001b[1mwandb\u001b[0m: Downloading large artifact minilm-arxiv:latest, 128.23MB. 6 files... \n",119"\u001b[34m\u001b[1mwandb\u001b[0m: 6 of 6 files downloaded. \n",120"Done. 0:0:0.1\n"121]122},123{124"data": {125"text/plain": [126"'./artifacts/minilm-arxiv:v1'"127]128},129"execution_count": 2,130"metadata": {},131"output_type": "execute_result"132}133],134"source": [135"import wandb\n",136"\n",137"run = wandb.init(project=\"arxiv-searching\")\n",138"\n",139"artifact = run.use_artifact(\n",140" 'jamesbriggs/arxiv-searching/minilm-arxiv:latest', type='model'\n",141")\n",142"artifact_dir = artifact.download()\n",143"artifact_dir"144]145},146{147"cell_type": "markdown",148"id": "467a9954-200f-49c4-8539-a0269e7c14d1",149"metadata": {},150"source": [151"Initialize it as a sentence transformer..."152]153},154{155"cell_type": "code",156"execution_count": 6,157"id": "7af9d2be-1144-4ef4-88f3-43b57371dda0",158"metadata": {},159"outputs": [160{161"data": {162"text/plain": [163"SentenceTransformer(\n",164" (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel \n",165" (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})\n",166")"167]168},169"execution_count": 6,170"metadata": {},171"output_type": "execute_result"172}173],174"source": [175"from sentence_transformers import models, SentenceTransformer\n",176"import torch\n",177"\n",178"minilm = models.Transformer(artifact_dir)\n",179"pooling = models.Pooling(\n",180" minilm.get_word_embedding_dimension(),\n",181" pooling_mode_mean_tokens=True\n",182")\n",183"\n",184"model = SentenceTransformer(\n",185" modules=[minilm, pooling],\n",186" device='cuda:1' if torch.cuda.is_available() else 'cpu'\n",187")\n",188"model"189]190},191{192"cell_type": "markdown",193"id": "875eba46",194"metadata": {},195"source": [196"Let's create an initial query and matching context."197]198},199{200"cell_type": "code",201"execution_count": 8,202"id": "667760a0-4e86-43c3-b5ee-00e07104e91c",203"metadata": {},204"outputs": [],205"source": [206"query = \"Ginzburg Landau theory for d-wave pairing and fourfold symmetric vortex\\n core structure\"\n",207"\n",208"xq = model.encode(query)"209]210},211{212"cell_type": "code",213"execution_count": 7,214"id": "58dcd6e2-d699-42a5-b0a0-6f5946a27f44",215"metadata": {},216"outputs": [],217"source": [218"c = \" The Ginzburg Landau theory for d_{x^2-y^2}-wave superconductors is\\nconstructed, by starting from the Gor'kov equation with including correction\\nterms up to the next order of ln(T_c/T). Some of the non-local correction terms\\nare found to break the cylindrical symmetry and lead to the fourfold symmetric\\ncore structure, reflecting the internal degree of freedom in the pair\\npotential. Using this extended Ginzburg Landau theory, we investigate the\\nfourfold symmetric structure of the pair potential, current and magnetic field\\naround an isolated single vortex, and clarify concretely how the vortex core\\nstructure deviates from the cylindrical symmetry in the d_{x^2-y^2}-wave\\nsuperconductors.\\n\""219]220},221{222"cell_type": "code",223"execution_count": 9,224"id": "fc26902e-a4c8-4ebc-a39e-4dbd653897b2",225"metadata": {},226"outputs": [],227"source": [228"cc = model.encode(c)"229]230},231{232"cell_type": "markdown",233"id": "7fa96ac5-0a23-4eb8-b8c9-56d3656caca5",234"metadata": {},235"source": [236"Let's do a quick sanity check:"237]238},239{240"cell_type": "code",241"execution_count": 10,242"id": "ff071de5-5b47-4525-b0b6-c1e904021631",243"metadata": {},244"outputs": [],245"source": [246"from sentence_transformers.util import cos_sim"247]248},249{250"cell_type": "code",251"execution_count": 11,252"id": "5229398f-c638-4bfd-9317-84c3191da274",253"metadata": {},254"outputs": [255{256"data": {257"text/plain": [258"tensor([[0.6709]])"259]260},261"execution_count": 11,262"metadata": {},263"output_type": "execute_result"264}265],266"source": [267"cos_sim(cc, xq)"268]269},270{271"cell_type": "code",272"execution_count": 12,273"id": "d6182e2c-edd1-419f-ad9d-d0eb278e3154",274"metadata": {},275"outputs": [],276"source": [277"r = \"something completely random that has nothing to do with the query\"\n",278"rr = model.encode(r)"279]280},281{282"cell_type": "code",283"execution_count": 13,284"id": "dfac7760-c2c9-46c7-bf31-e84f8dfee782",285"metadata": {},286"outputs": [287{288"data": {289"text/plain": [290"tensor([[-0.0784]])"291]292},293"execution_count": 13,294"metadata": {},295"output_type": "execute_result"296}297],298"source": [299"cos_sim(rr, xq)"300]301},302{303"cell_type": "markdown",304"id": "f63bd5d5-bdd1-4345-8e7b-fa09bae30f06",305"metadata": {},306"source": [307"Looks good the cosine similarity between the actual pair is high, whereas between something complete random it is not. Now let's begin performing actual queries:"308]309},310{311"cell_type": "code",312"execution_count": 15,313"id": "d674d177-4121-4614-8c64-2d4d04575de6",314"metadata": {},315"outputs": [316{317"data": {318"text/plain": [319"{'matches': [{'id': '1304.4032',\n",320" 'metadata': {'abstract': ' A procedure to derive the '\n",321" 'Ginzburg-Landau (GL) theory from the '\n",322" 'multiband BCS\\n'\n",323" 'Hamiltonian is developed in a general '\n",324" 'case with an arbitrary number of '\n",325" 'bands\\n'\n",326" 'and arbitrary interaction matrix. It '\n",327" \"combines the standard Gor'kov \"\n",328" 'truncation\\n'\n",329" 'and a subsequent reconstruction in '\n",330" 'order to match accuracies of the '\n",331" 'obtained\\n'\n",332" 'terms. This reconstruction recovers '\n",333" 'the phenomenological GL theory as '\n",334" 'obtained\\n'\n",335" 'from the Landau model of phase '\n",336" 'transitions but offers explicit '\n",337" 'microscopic\\n'\n",338" 'expressions for the relevant '\n",339" 'parameters. Detailed calculations are '\n",340" 'presented\\n'\n",341" 'for a three-band system treated as a '\n",342" 'prototype multiband superconductor. It '\n",343" 'is\\n'\n",344" 'demonstrated that the symmetry in the '\n",345" 'coupling matrix may lead to the '\n",346" 'chiral\\n'\n",347" 'ground state with the phase '\n",348" 'frustration, typical for systems with '\n",349" 'broken\\n'\n",350" 'time-reversal symmetry.\\n'},\n",351" 'score': 0.653086662,\n",352" 'sparseValues': {},\n",353" 'values': []},\n",354" {'id': '0710.3482',\n",355" 'metadata': {'abstract': ' We succeed to build up a '\n",356" 'straightforward theoretical model for '\n",357" 'spin-triplet\\n'\n",358" 'p-wave superconductors by introducing '\n",359" 'in Ginzburg-Landau theory a second '\n",360" 'order\\n'\n",361" 'parameter and a suitable interaction '\n",362" 'between the two mean fields.\\n'},\n",363" 'score': 0.591999769,\n",364" 'sparseValues': {},\n",365" 'values': []},\n",366" {'id': '1110.1593',\n",367" 'metadata': {'abstract': ' We report a microscopic derivation '\n",368" 'of two-component Ginzburg-Landau (GL)\\n'\n",369" 'field theory and the conditions of its '\n",370" 'validity in two-band superconductors. '\n",371" 'We\\n'\n",372" 'also investigate the conditions when '\n",373" 'microscopically derived or\\n'\n",374" 'phenomenological GL models fail and '\n",375" 'one should resort to a microscopic\\n'\n",376" 'description.\\n'\n",377" ' We show that besides being directly '\n",378" 'applicable at elevated temperatures, '\n",379" 'a\\n'\n",380" 'version of a minimal two-component GL '\n",381" 'theory in certain cases also gives\\n'\n",382" 'accurate description of certain '\n",383" 'aspects of a two-band system even '\n",384" 'substantially\\n'\n",385" 'far from $T_c$. This shows that '\n",386" 'two-component GL model can be used '\n",387" 'for\\n'\n",388" 'addressing a wide range of questions '\n",389" 'in multiband systems, in particular '\n",390" 'vortex\\n'\n",391" 'physics and magnetic response. We also '\n",392" 'argue that single Ginzburg-Landau\\n'\n",393" 'parameter cannot in general '\n",394" 'characterize magnetic response of '\n",395" 'multiband\\n'\n",396" 'systems.\\n'},\n",397" 'score': 0.577286422,\n",398" 'sparseValues': {},\n",399" 'values': []},\n",400" {'id': '0707.1736',\n",401" 'metadata': {'abstract': ' The one-dimensional Ginzburg-Landau '\n",402" '(GL) Equation is considered. We use '\n",403" 'the\\n'\n",404" 'recently developed extended '\n",405" 'F-expansion method to obtain spiral '\n",406" 'wave solution\\n'\n",407" 'of one-dimensional GL Equation.\\n'},\n",408" 'score': 0.568699062,\n",409" 'sparseValues': {},\n",410" 'values': []},\n",411" {'id': '1111.1314',\n",412" 'metadata': {'abstract': ' We elaborate on boundary conditions '\n",413" 'for Ginzburg-Landau (GL) theory in '\n",414" 'the\\n'\n",415" 'case of external currents. We '\n",416" 'implement a self-consistent theory '\n",417" 'within the\\n'\n",418" 'finite element method (FEM) and '\n",419" 'present numerical results for a '\n",420" 'two-dimensional\\n'\n",421" 'rectangular geometry. We emphasize '\n",422" 'that our approach can in principle '\n",423" 'also be\\n'\n",424" 'used for general geometries in '\n",425" 'three-dimensional superconductors.\\n'},\n",426" 'score': 0.561762691,\n",427" 'sparseValues': {},\n",428" 'values': []}],\n",429" 'namespace': ''}"430]431},432"execution_count": 15,433"metadata": {},434"output_type": "execute_result"435}436],437"source": [438"xc = index.query(vector=xq.tolist(), top_k=5, include_metadata=True)\n",439"xc"440]441},442{443"cell_type": "markdown",444"id": "b997bd08-c5f9-4f31-b571-5c5f5954e8dc",445"metadata": {},446"source": [447"We get a perfect result with the top item being the specific abstract for the given title. We can try with some more..."448]449},450{451"cell_type": "code",452"execution_count": 17,453"id": "1092133f-fe89-4234-95b6-c1852073e7de",454"metadata": {},455"outputs": [456{457"data": {458"text/plain": [459"{'matches': [{'id': '2105.05674',\n",460" 'metadata': {'abstract': ' Game developers benefit from '\n",461" 'availability of custom game genres '\n",462" 'when doing\\n'\n",463" 'game market analysis. This information '\n",464" 'can help them to spot opportunities '\n",465" 'in\\n'\n",466" 'market and make them more successful '\n",467" 'in planning a new game. In this paper '\n",468" 'we\\n'\n",469" 'find good classifier for predicting '\n",470" 'category of a game. Prediction is '\n",471" 'based on\\n'\n",472" 'description and title of a game. We '\n",473" 'use 2443 iOS App Store games as data '\n",474" 'set to\\n'\n",475" 'generate a document-term matrix. To '\n",476" 'reduce the curse of dimensionality we '\n",477" 'use\\n'\n",478" 'Latent Semantic Indexing, which, '\n",479" 'reduces the term dimension to '\n",480" 'approximately\\n'\n",481" '1/9. Support Vector Machine supervised '\n",482" 'learning model is fit to '\n",483" 'pre-processed\\n'\n",484" 'data. Model parameters are optimized '\n",485" 'using grid search and 20-fold cross\\n'\n",486" 'validation. Best model yields to 77% '\n",487" 'mean accuracy or roughly 70% accuracy '\n",488" 'with\\n'\n",489" '95% confidence. Developed classifier '\n",490" 'has been used in-house to assist '\n",491" 'games\\n'\n",492" 'market research.\\n'},\n",493" 'score': 0.696358502,\n",494" 'sparseValues': {},\n",495" 'values': []},\n",496" {'id': '2101.00992',\n",497" 'metadata': {'abstract': ' We develop methods to formally '\n",498" 'describe and compare games, in order '\n",499" 'to probe\\n'\n",500" 'questions of game structure and '\n",501" 'design, and as a stepping stone to '\n",502" 'predicting\\n'\n",503" 'player behavior from design patterns. '\n",504" 'We define a grammar-like formalism to\\n'\n",505" 'describe finite discrete games without '\n",506" 'hidden information, allowing for\\n'\n",507" 'randomness, and mixed sequential and '\n",508" 'simultaneous play. We make minimal\\n'\n",509" 'assumptions about the form or content '\n",510" 'of game rules or user interface. The\\n'\n",511" 'associated game trees resemble hybrid '\n",512" 'extensive- and strategic-form games, '\n",513" 'in\\n'\n",514" 'the game theory sense. By transforming '\n",515" 'and comparing game trees, we develop\\n'\n",516" 'equivalence relations on the space of '\n",517" 'game systems, which equate games that\\n'\n",518" 'give players the same meaningful '\n",519" 'agency. We bring these together to '\n",520" 'suggest a\\n'\n",521" 'method to measure distance between '\n",522" 'games, insensitive to cosmetic '\n",523" 'variations in\\n'\n",524" 'the game logic descriptions.\\n'},\n",525" 'score': 0.654641,\n",526" 'sparseValues': {},\n",527" 'values': []},\n",528" {'id': '1912.01876',\n",529" 'metadata': {'abstract': ' Many problems can be viewed as '\n",530" 'games, where one or more agents try to '\n",531" 'ensure\\n'\n",532" 'that certain objectives hold no matter '\n",533" 'the behavior from the environment and\\n'\n",534" 'other agents. In recent years, a '\n",535" 'number of logical formalisms have '\n",536" 'been\\n'\n",537" 'proposed for specifying games among '\n",538" 'which the Game Description Language '\n",539" '(GDL)\\n'\n",540" 'was established as the official '\n",541" 'language for General Game Playing. '\n",542" 'Although\\n'\n",543" 'numbers are recurring in games, the '\n",544" 'description of games with numerical\\n'\n",545" 'features in GDL requires the '\n",546" 'enumeration from all possible numeric '\n",547" 'values and\\n'\n",548" 'the relation among them. Thereby, in '\n",549" 'this paper, we introduce the Game\\n'\n",550" 'Description Logic with Integers (GDLZ) '\n",551" 'to describe games with numerical\\n'\n",552" 'variables, numerical parameters, as '\n",553" 'well as to perform numerical '\n",554" 'comparisons.\\n'\n",555" 'We compare our approach with GDL and '\n",556" 'show that when describing the same '\n",557" 'game,\\n'\n",558" 'GDLZ is more compact.\\n'},\n",559" 'score': 0.652700245,\n",560" 'sparseValues': {},\n",561" 'values': []},\n",562" {'id': '1711.10860',\n",563" 'metadata': {'abstract': ' Game semantics is a rich and '\n",564" 'successful class of denotational '\n",565" 'models for\\n'\n",566" 'programming languages. Most game '\n",567" 'models feature a rather intuitive '\n",568" 'setup, yet\\n'\n",569" 'surprisingly difficult proofs of such '\n",570" 'basic results as associativity of\\n'\n",571" 'composition of strategies. We set out '\n",572" 'to unify these models into a basic\\n'\n",573" 'abstract framework for game semantics, '\n",574" 'game settings. Our main contribution '\n",575" 'is\\n'\n",576" 'the generic construction, for any game '\n",577" 'setting, of a category of games and\\n'\n",578" 'strategies. Furthermore, we extend the '\n",579" 'framework to deal with innocence, and\\n'\n",580" 'prove that innocent strategies form a '\n",581" 'subcategory. We finally show that our\\n'\n",582" 'constructions cover many concrete '\n",583" 'cases, mainly among the early models '\n",584" 'and the\\n'\n",585" 'very recent sheaf-based ones.\\n'},\n",586" 'score': 0.640093,\n",587" 'sparseValues': {},\n",588" 'values': []},\n",589" {'id': '2207.11690',\n",590" 'metadata': {'abstract': ' Machine sound classification has '\n",591" 'been one of the fundamental tasks of '\n",592" 'music\\n'\n",593" 'technology. A major branch of sound '\n",594" 'classification is the classification '\n",595" 'of\\n'\n",596" 'music genres. However, though covering '\n",597" 'most genres of music, existing music\\n'\n",598" 'genre datasets often do not contain '\n",599" 'fine-grained labels that indicate the\\n'\n",600" 'detailed sub-genres of music. In '\n",601" 'consideration of the consistency of '\n",602" 'genres of\\n'\n",603" 'songs in a mixtape or in a DJ (live) '\n",604" 'set, we have collected and annotated '\n",605" 'a\\n'\n",606" 'dataset of house music that provide 4 '\n",607" 'sub-genre labels, namely future '\n",608" 'house,\\n'\n",609" 'bass house, progressive house and '\n",610" 'melodic house. Experiments show that '\n",611" 'our\\n'\n",612" 'annotations well exhibit the '\n",613" 'characteristics of different '\n",614" 'categories. Also, we\\n'\n",615" 'have built baseline models that '\n",616" 'classify the sub-genre based on the\\n'\n",617" 'mel-spectrograms of a track, achieving '\n",618" 'strongly competitive results. '\n",619" 'Besides,\\n'\n",620" 'we have put forward a few application '\n",621" 'scenarios of our dataset and baseline\\n'\n",622" 'model, with a simulated sci-fi tunnel '\n",623" 'as a short demo built and rendered in '\n",624" 'a\\n'\n",625" '3D modeling software, with the colors '\n",626" 'of the lights automated by the output '\n",627" 'of\\n'\n",628" 'our model.\\n'},\n",629" 'score': 0.626893759,\n",630" 'sparseValues': {},\n",631" 'values': []}],\n",632" 'namespace': ''}"633]634},635"execution_count": 17,636"metadata": {},637"output_type": "execute_result"638}639],640"source": [641"query = \"classifier for genre categorization of a game\"\n",642"\n",643"xq = model.encode(query).tolist()\n",644"\n",645"xc = index.query(vector=xq, top_k=5, include_metadata=True)\n",646"xc"647]648},649{650"cell_type": "markdown",651"id": "cfccb607-11fa-4d5a-92d0-fd516c1efa1c",652"metadata": {},653"source": [654"Another great result, let's try one more..."655]656},657{658"cell_type": "code",659"execution_count": 19,660"id": "b43dd25b-0e1d-46c6-b1e9-89183615abb2",661"metadata": {},662"outputs": [663{664"data": {665"text/plain": [666"{'matches': [{'id': '2206.05555',\n",667" 'metadata': {'abstract': ' Multi-modal pre-training and '\n",668" 'knowledge discovery are two important '\n",669" 'research\\n'\n",670" 'topics in multi-modal machine '\n",671" 'learning. Nevertheless, none of '\n",672" 'existing works\\n'\n",673" 'make attempts to link knowledge '\n",674" 'discovery with knowledge guided '\n",675" 'multi-modal\\n'\n",676" 'pre-training. In this paper, we '\n",677" 'propose to unify them into a '\n",678" 'continuous\\n'\n",679" 'learning framework for mutual '\n",680" 'improvement. Taking the open-domain '\n",681" 'uni-modal\\n'\n",682" 'datasets of images and texts as input, '\n",683" 'we maintain a knowledge graph as the\\n'\n",684" 'foundation to support these two tasks. '\n",685" 'For knowledge discovery, a '\n",686" 'pre-trained\\n'\n",687" 'model is used to identify cross-modal '\n",688" 'links on the graph. For model\\n'\n",689" 'pre-training, the knowledge graph is '\n",690" 'used as the external knowledge to '\n",691" 'guide\\n'\n",692" 'the model updating. These two steps '\n",693" 'are iteratively performed in our '\n",694" 'framework\\n'\n",695" 'for continuous learning. The '\n",696" 'experimental results on MS-COCO and '\n",697" 'Flickr30K with\\n'\n",698" 'respect to both knowledge discovery '\n",699" 'and the pre-trained model validate '\n",700" 'the\\n'\n",701" 'effectiveness of our framework.\\n'},\n",702" 'score': 0.777464151,\n",703" 'sparseValues': {},\n",704" 'values': []},\n",705" {'id': '2102.04883',\n",706" 'metadata': {'abstract': ' This is an introductory '\n",707" 'machine-learning course specifically '\n",708" 'developed with\\n'\n",709" 'STEM students in mind. Our goal is to '\n",710" 'provide the interested reader with '\n",711" 'the\\n'\n",712" 'basics to employ machine learning in '\n",713" 'their own projects and to familiarize\\n'\n",714" 'themself with the terminology as a '\n",715" 'foundation for further reading of the\\n'\n",716" 'relevant literature. In these lecture '\n",717" 'notes, we discuss supervised,\\n'\n",718" 'unsupervised, and reinforcement '\n",719" 'learning. The notes start with an '\n",720" 'exposition of\\n'\n",721" 'machine learning methods without '\n",722" 'neural networks, such as principle '\n",723" 'component\\n'\n",724" 'analysis, t-SNE, clustering, as well '\n",725" 'as linear regression and linear\\n'\n",726" 'classifiers. We continue with an '\n",727" 'introduction to both basic and '\n",728" 'advanced\\n'\n",729" 'neural-network structures such as '\n",730" 'dense feed-forward and conventional '\n",731" 'neural\\n'\n",732" 'networks, recurrent neural networks, '\n",733" 'restricted Boltzmann machines,\\n'\n",734" '(variational) autoencoders, generative '\n",735" 'adversarial networks. Questions of\\n'\n",736" 'interpretability are discussed for '\n",737" 'latent-space representations and using '\n",738" 'the\\n'\n",739" 'examples of dreaming and adversarial '\n",740" 'attacks. The final section is '\n",741" 'dedicated to\\n'\n",742" 'reinforcement learning, where we '\n",743" 'introduce basic notions of value '\n",744" 'functions and\\n'\n",745" 'policy learning.\\n'},\n",746" 'score': 0.728711367,\n",747" 'sparseValues': {},\n",748" 'values': []},\n",749" {'id': '2007.01126',\n",750" 'metadata': {'abstract': ' Multi-task learning (MTL) optimizes '\n",751" 'several learning tasks simultaneously '\n",752" 'and\\n'\n",753" 'leverages their shared information to '\n",754" 'improve generalization and the '\n",755" 'prediction\\n'\n",756" 'of the model for each task. Auxiliary '\n",757" 'tasks can be added to the main task '\n",758" 'to\\n'\n",759" 'ultimately boost the performance. In '\n",760" 'this paper, we provide a brief review '\n",761" 'on\\n'\n",762" 'the recent deep multi-task learning '\n",763" '(dMTL) approaches followed by methods '\n",764" 'on\\n'\n",765" 'selecting useful auxiliary tasks that '\n",766" 'can be used in dMTL to improve the\\n'\n",767" 'performance of the model for the main '\n",768" 'task.\\n'},\n",769" 'score': 0.725683272,\n",770" 'sparseValues': {},\n",771" 'values': []},\n",772" {'id': '1703.05298',\n",773" 'metadata': {'abstract': ' This report provides an introduction '\n",774" 'to some Machine Learning tools within\\n'\n",775" 'the most common development '\n",776" 'environments. It mainly focuses on '\n",777" 'practical\\n'\n",778" 'problems, skipping any theoretical '\n",779" 'introduction. It is oriented to both\\n'\n",780" 'students trying to approach Machine '\n",781" 'Learning and experts looking for new\\n'\n",782" 'frameworks.\\n'},\n",783" 'score': 0.724752188,\n",784" 'sparseValues': {},\n",785" 'values': []},\n",786" {'id': '2210.11024',\n",787" 'metadata': {'abstract': ' Recently self supervised learning '\n",788" 'has seen explosive growth and use in\\n'\n",789" 'variety of machine learning tasks '\n",790" 'because of its ability to avoid the '\n",791" 'cost of\\n'\n",792" 'annotating large-scale datasets.\\n'\n",793" ' This paper gives an overview for '\n",794" 'best self supervised learning '\n",795" 'approaches for\\n'\n",796" 'multimodal learning. The presented '\n",797" 'approaches have been aggregated by '\n",798" 'extensive\\n'\n",799" 'study of the literature and tackle the '\n",800" 'application of self supervised '\n",801" 'learning\\n'\n",802" 'in different ways. The approaches '\n",803" 'discussed are cross modal generation, '\n",804" 'cross\\n'\n",805" 'modal pretraining, cyclic translation, '\n",806" 'and generating unimodal labels in '\n",807" 'self\\n'\n",808" 'supervised fashion.\\n'},\n",809" 'score': 0.722871125,\n",810" 'sparseValues': {},\n",811" 'values': []}],\n",812" 'namespace': ''}"813]814},815"execution_count": 19,816"metadata": {},817"output_type": "execute_result"818}819],820"source": [821"query = \"the best pre-training approaches for multi-modal machine learning\"\n",822"\n",823"xq = model.encode(query).tolist()\n",824"\n",825"xc = index.query(vector=xq, top_k=5, include_metadata=True)\n",826"xc"827]828},829{830"cell_type": "markdown",831"id": "5f1c9ebd-8c6e-48b4-85d5-cd0f1695ed8b",832"metadata": {},833"source": [834"In this case it seems the best result is in position *5* (still not bad out of 2M+ abstracts) and the top result seems relevant but might not quite answer our query."835]836},837{838"attachments": {},839"cell_type": "markdown",840"id": "67dfc5ea",841"metadata": {},842"source": [843"Once we're finished, we delete the index to save resources."844]845},846{847"cell_type": "code",848"execution_count": null,849"id": "144fd999",850"metadata": {},851"outputs": [],852"source": [853"pinecone.delete_index(index_id)"854]855},856{857"attachments": {},858"cell_type": "markdown",859"id": "14fa79c2",860"metadata": {},861"source": [862"---"863]864}865],866"metadata": {867"environment": {868"kernel": "python3",869"name": "common-cu110.m95",870"type": "gcloud",871"uri": "gcr.io/deeplearning-platform-release/base-cu110:m95"872},873"kernelspec": {874"display_name": "Python 3.10.7 64-bit",875"language": "python",876"name": "python3"877},878"language_info": {879"codemirror_mode": {880"name": "ipython",881"version": 3882},883"file_extension": ".py",884"mimetype": "text/x-python",885"name": "python",886"nbconvert_exporter": "python",887"pygments_lexer": "ipython3",888"version": "3.10.9"889},890"vscode": {891"interpreter": {892"hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"893}894}895},896"nbformat": 4,897"nbformat_minor": 5898}899