{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "# HDFS Service\n",
    "\n",
    "The platform provides a managed Hadoop File System (HDFS) for big data storage and computing purposes.\n",
    "\n",
    "HDFS name node web interface is available through the launcher. Please enter your UT credentials to access the interface.\n",
    "\n",
    "You can use standard CLI commands to perform HDFS operations."
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "hadoop fs -copyFromLocal /data/public/readme.md"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "copyFromLocal: `readme.md': File exists\n"
     ]
    },
    {
     "output_type": "error",
     "ename": "",
     "evalue": "1",
     "traceback": []
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "hadoop fs -ls"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Found 2 items\n",
      "drwxr-xr-x   - jovyan supergroup          0 2021-01-15 13:53 .sparkStaging\n",
      "-rw-r--r--   1 jovyan supergroup        583 2021-01-15 10:56 readme.md\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "source": [
    "hadoop fs -cat readme.md"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "# Public Folder\n",
      "\n",
      "This **read-only** folder contains public files shared with all users. \n",
      "\n",
      "The main purpose of the folder is to share common datasets, such as base vector and raster maps.\n",
      "\n",
      "If you need such datasets, please send an e-mail to <s.girgin@utwente.nl> with the following information:\n",
      "\n",
      "- Name of the dataset\n",
      "- URL address for download\n",
      "\n",
      "Self-service portal will be available soon to request datasets.\n",
      "\n",
      "You can find useful information on the platform, such as lists of available packages, feature tests, configuration files, and benchmark results, under the `platform` folder."
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "For more information please refer to the [Apache Hadoop Documentation](https://hadoop.apache.org/docs/current/)"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "source": [],
   "outputs": [],
   "metadata": {}
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}