Compare commits

..

No commits in common. '04db1e01466734d18da434ae3b9f48bb866d69ed' and '7ab7f8f493bc8fc079162e0feea3cbbd2be8f83f' have entirely different histories.

1
.gitignore vendored

@ -1 +0,0 @@
.ipynb_checkpoints

@ -1,106 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "3b5fe448-682e-4168-9f0f-ddd4e147b59d",
"metadata": {},
"source": [
"# Challenge I: Search and process data from DIAS (Copernicus Data and Information Access Services)\n",
"*WRS jupyter hackathon, 30th of June 2021*, R. Rietbroek\n",
"\n",
"The overall goal of this challenge is threefold:\n",
"1. Learn how to establish a connection with a DIAS server and execute a search request\n",
"2. Download a dataset\n",
"3. Apply a post-processing operation on such a dataset\n",
"\n",
"DIAS currently consists of 5 different data providers, which hosts a variety of data and provide access to it:\n",
"\n",
"<a href=\"https://creodias.eu/\" target=\"_blank\">\n",
" <img src=\"https://eo4society.esa.int/wp-content/uploads/2018/11/creodias_logo-e1543314673171.png\" />\n",
"</a>\n",
"\n",
"<a href=\"https://mundiwebservices.com\" target=\"_blank\">\n",
" <img src=\"https://eo4society.esa.int/wp-content/uploads/2018/11/mundi_logo-e1543314714627.png\" />\n",
"</a>\n",
"\n",
"<a href=\"https://www.onda-dias.eu/cms/\" target=\"_blank\">\n",
" <img src=\"https://eo4society.esa.int/wp-content/uploads/2018/11/onda_logo-e1543314731347.png\" />\n",
"</a>\n",
"\n",
"<a href=\"https://sobloo.eu\" target=\"_blank\">\n",
" <img src=\"https://eo4society.esa.int/wp-content/uploads/2018/11/sobloo_logo-e1543314744507.png\" />\n",
"</a>\n",
"\n",
"<a href=\"https://wekeo.eu\" target=\"_blank\">\n",
" <img src=\"https://eo4society.esa.int/wp-content/uploads/2018/11/wekeo_logo-e1543314756229.png\" />\n",
"</a>\n",
"\n",
"The different providers generally provide a graphical website to search for and access data. However, they also provide scriptable ways which allow automated access (so-called [REST-API's](https://en.wikipedia.org/wiki/Representational_state_transfer) in webserver speak). In essence, these API's allow the server to be passed additional arguments (append by using `?parameter=Value1&parameter2=Value2` which can be interpreted and processed by the server and a response can be send back. For example [https://catalogue.onda-dias.eu/opensearch/OpenSearch?instrumentShortName=MSI](https://catalogue.onda-dias.eu/opensearch/OpenSearch?instrumentShortName=MSI), will return a machine readable xml document containing the search hits for the term `MSI`. \n",
"\n",
"In this challenge, you will try to make use of the automated way to search for and access the data, in order to facilitate automated scripts. \n",
"\n",
"Unfortunately, the way to access the servers is not standardized and searches and download requests may take several forms depending on the provider. Luckily there is [progress on a python module called eodag](https://pypi.org/project/eodag/) which provides a more uniform way of accessing the the providers. You're encouraged to make use of that package."
]
},
{
"cell_type": "markdown",
"id": "ecc297bd-ce71-4099-8674-be4262138eda",
"metadata": {},
"source": [
"## Challenge statement\n",
"Supplement this notebook with functionality which (1) allow search queries to the datasets, (2) Downloading an appropriate subset of the data, (3) create a simple visualization of the downloaded data\n",
"\n",
"## Tips and tricks\n",
"* Try to find interesting datasets on the graphical webinterfaces of the servers, and see if those can also be found in a programmatic way.\n",
"* Try out with 'light' datasets first in order to avoid repeated downloading of large files. Try to see if *subsetting* is possible (downloading only parts of the dataset)\n",
"* There are many python code snippets in the crib's folder `public/resources/Python-Data-Science-Handbook` which may be of use\n",
"* From a security standpoint: Don't hardcode usernames and passwords in your jupyter notebook. For example, you can query for the user's input (see below) or use separate files which contain confidential information"
]
},
{
"cell_type": "code",
"execution_count": 9,
"id": "34d416dd-a9bc-48e1-8cec-8d443a3373a0",
"metadata": {},
"outputs": [
{
"name": "stdin",
"output_type": "stream",
"text": [
"Please enter username roelof\n",
"Please enter password ·····\n"
]
}
],
"source": [
"#example how to query the user for sensitive information\n",
"from getpass import getpass\n",
"credentials={}\n",
"credentials[\"user\"]=input(\"Please enter username\")\n",
"credentials[\"pass\"]=getpass(\"Please enter password\")\n",
"# Note that this information is 'volatile': when the notebook shuts down the values of credentials are lost and not stored in the notebook"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

File diff suppressed because one or more lines are too long

@ -1,48 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "f6b37abb-e3fe-43d0-8527-783a84f457de",
"metadata": {},
"source": [
"# Challenge III Open Challenge pick your own topic and assemble a team \n",
"![open challenge](https://www.nairaland.com/attachments/1951313_openchallenge_jpeg486cbc900da03112763d6a527d81e2d6)\n",
"\n",
"There may be projects where you're looking for additional help in solving some hurdles. In this challenge you can pick a topic of your liking but we do like to ask you to find at least 1 team member who can join you in your team. \n",
"\n",
"## Objectives \n",
"\n",
"1. Try to simplify the problem at hand \n",
"\n",
"2. Find a solution\n",
"\n",
"3. Present your results on a Jupyter notebook. \n",
"\n",
"## What we ask of you if you go for this topic\n",
"* A concise problem definition (simplify the problem as much as possible)\n",
"* Find a team to help you\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

@ -1,40 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "4fc5da8a-9e7a-4571-a286-c89cf81b2283",
"metadata": {},
"source": [
"# Welcome to the WRS hackathon\n",
"\n",
"## Goals & expectations\n",
"\n",
"## Schedule\n",
"\n",
"\n",
"\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading…
Cancel
Save