DAD - Distributed and Cloud Applications Development - Dezvoltarea Aplicatiilor Distribuite si Cloud

Mission

Technological transfer from university to the students of practical and theoretical issues related with Distributed and Cloud Applications Development.

* Message regarding Covid-19:

Please follow the info and announcements on:

Starting from now (February 22-nd, 2021, old date: March 9-th, 2020), depending on the evolution / involution of the phenomenon at national and European level, the activities will be held on the tele-working platform Zoom.

In order to have effective communications, we will use the following state of the art tools:

1. Zoom for the lectures and seminars/labs - unless the Java teams professor is not communicating different link for his labs group: https://ase.zoom.us/j/82446395333?pwd=R3pyQzBMRTNrREp6QW9NQjhVZUxxdz09

2. Slack channel for async professional communications -  Java-2021: https://dad-ie-2021.slack.com/ - Joinable Link (please use institutional e-mail address and full name): https://join.slack.com/t/newworkspace-gc53972/shared_invite/zt-mtvzh4vl-OJZHc6wEnSAZC6V0Fpw3YA 

* For the period February-June 2021, we have the following info: Group 1099 - on Mon, from 17:30 | Group 1098 - on Mon, from 19:30 | Lecture - on Thu, from 19:30.

We have used this platform - Zoom including in the university rooms (old link: https://zoom.us/j/4916362091?pwd=d-38wffyh1E), because we had to compensate the poor image quality from the video projector and as preventive "training" for any unforeseen situations.

The Cyber Security Master Program: http://www.ism.ase.ro has been upgraded on Tue, March 9-th, 2020, the Zoom platform, so that the working session will be extended from 40 minutes to 12 hours. The presence is mandatory both in the course, as well as in the seminar / lab, in Zoom virtual sessions, so please use the real name in Zoom when connecting from home. As best practice, it is better to connect in Zoom from a smart phone or tablet and type the code into the laptop, because it is lower probability to have technical issues with the intalled drivers for the voice and speakers in tablet/smart phones, than in the laptops. The Sakai platform: http://37.120.250.22:8020/portal may be used as replacement for www.online.ase.ro if necessary.

Lectures & Labs

No.  Lecture 
PDF
Lecture Description Sources for the 
Lecture
Lab Description Sources for the
Lab 
-

INTRO - Distributed Applications Development administrative issues

1.

RECAP: Java Multi-threading mechanisms, java.util.concurrent package in samples for HPC - High Performance Computing samples | IPC - Inter-Process Communication in Linux using C/C++, Linux C POSIX Multi-Threading (PThread), Java Multi-threading and Parallel Computing Intro. Multi-tenancy via multi-process or multi-threading.

For the Java intro full details (RECAP sections), please see the lectures and labs of Multi-paradigm Programming - Java @ http://acs.ase.ro/java and first three activities

For Parallel Computing (incl. C OpenMP) intro please see lectures and labs details @ http://acs.ase.ro/parallel-processing

Distributed vs Parallel Computing Systems
* HTC (High Throughput Computing) vs. HPC (High Performance Computing)

RECAPJava Multi-threading mechanisms, java.util.concurrent package in samples for HPC - High Performance Computing samples 
2.

RECAP: TCP/IP Network Concepts - OSI Model, IP v4 vs. v6, UDP headers, NAT, PAT, Network Programming in Java - UDP, SNMP client over UDP sample - network monitoring.

ANT, Maven, Gradle, OS VM vs. Docker - virtualization, DevOps Jenkins Intro

Intro in Java Reflection & Annotations, JNI - Java Native Interface, Java Exception mechanism, Source Code Design Patterns - Singleton.

RECAP: Network UDP - Java UDP Network programming using Socket, SNMP Client sample - simple and with library snmp4j
3.

RECAP: Java 8+ Lamda with Multi-threading, TCP/IP Network Concepts - Network Programming in Java - UDP & TCP with multi-threading, IP, TCP headers, TCP state machine, NAT, PAT, Network Programming in Java - TCP

RECAP: Java 8+ Lambda - Network HTTP-Multi-threading and TCP with lambda Example - web server 
4.

Java Servlet Intro - Apache Tomcat web server with servlet container distribution, Session Tracking, HTTP Requests and Responses, Cookies, HTTP 1.1 

Cloud PaaS Intro: Deployment of Servlet into Google App Engine - PaaS Cloud.

HTTP/1.1 understanding with Wireshark - Java Servlet Sample with annotation instead web.xml descriptor, deployment in Apache Tomcat 9 + processing a HTML form with Java Servlet.

5.

Java Server Pages - JSP Intro - Apache Tomcat web server with Servlet/JSP container distribution, JSP Architecture, Directives and Actions, Objects, Tag-Libs, Beans, MVC - Model View Controller Type 1 and 2 Intro

Servlet and JSP are part of Web curriculum per se, and they are requred in this disscipline as data gathering/Cloud Gateway for massive distributed and parallel computing in back-end servers: Cloud, GRID, Fog infrastructures 

Java Server Pages - JSP Sample deployment in Apache Tomcat 9 + processing a HTML form with JSP and SQLite/MongoDB access.

Local deployment into VMBox.

6.

RPC - Remote Procedure Call and Java RMI - Remote Method Invocation

Java RMI sample for adding two integer vectors.
7.

SOA (Service Oriented Architectures) WebServices SOAP - Simple Objects Access Protocol

Process Migrations vs. Data Migrations within a Distributed System | TCP Socket example for passing business logic and data migration

SOA WebServices SOAP - Java JAX-WS in Eclipse (XML Recap)

Stand-alone client and server application

JAX-WS Web Server app

8.

RESTlful Services  vs. SOA | JAX-RS with HTTP/2

Difference between HTTP/1.1 and HTTP/2.0 with Wireshark network packets analyzer

HTTP/2 Client API with JDK11+

Open MPI

Cloud IaaS Intro: Deployment of OpenMPI programs in Amazon EC2

RESTful service in Java JAX-RS in Eclipse (JSON Recap)
T9.

TEST on PC

TEST on PC
9.

node.js INTRO/RECAP

Cloud FaaS - Functions as a Service / AWS Lamda Sever-less Programming Example vs. Fog Computing 

Angular.js vs. REACT vs. Apache Cordova + WebAssembly - Web/Mobile Front-End Development is NOT part (beyond) of this curriculum of this lecture - they should be part of Web or Mobile topic

Node.js JavaScript/ECMAScript - incl. WebServer & RESTful service in Node.js + WebSockets (Java/Node.js)
10.

Middleware & MoM: JMS vs. Apache Kafka

Middleware: Apache JMS and Kafka implementation for the Producer and Consumer Clients example
11.

Distributed Components: EJB - Enterprise Java Beans / Java | Jakarta EE platform vs. Actors in Akka 

Apache EJB 3.x and MDB - Message Driven Bean vs. Akka actors examples
12.

Intro n-Tier Architecture, Back-end Servers Clustering & Servers Infrastructures & (Native) Clouds - IaaS, PaaS, CaaS, SaaS, FaaS and Fog Computing

Infrastructures for Microservices / PODs and Containers as a Service Cloud - CaaS - Docker | Kubernetes

Spring Boot: JMS and components beans plus RESTful Services 

13.

14.

Spring Microservices in Action with Java/Kotlin and WASM - Web Assebmly as alternative to Java Applets technology

Spring End2End Microservice Example

E-Learning Solution & E-Framework Solutions

If necessary, the E-Learning solution platform is based on SAKAI CLE, powered by Sakai Project and it is provided by ISM - IT | Cyber Security Master program: http://ism.ase.ro and ACS - Applied Computer Science And Cyber Security Team: http://acs.ase.roSAKAI e-Learning System is available @ http://37.120.250.22:8020/portal 

E-Framework solutions for Distributed Applications Development (DAD) lectures and labs are stored in Git. The students may get them into the Oracle VM VirtualBox client virtual machine or Docker Container Image, and they are provided by Cyber Security Master program - http://ism.ase.ro
The details are the following:

1. The source code for the lectures and labs is in Git @: https://github.com/critoma/dad 

2. The Linux Ubuntu 16.04 LTS OS VM as Oracle VirtualBox OVA Appliance for download - with GUI and Eclipse 2018-09 & IntelliJ IDEA 2018 as Java-IDE - 14GB @: 

https://drive.google.com/open?id=1kKZChf2qbYME9PtO3Gu9ZkecRoNykWsD

3. Partial alternative to Oracle VirtualBox OVA, Docker Container Image with Linux Ubuntu 16.04 LTS for download - without GUI - 4GB @: 

https://hub.docker.com/r/critoma/ubuntu-java-node-py-dev/ 

Beyond the GUI and Eclipse & IntelliJ Java-IDE differences, both, the Oracle VirtualBox OVA appliance and the Docker Container Image contain:

Linux OS Ubuntu 16.04 x64 Xenial with: Java - Oracle JDK 8 and 9 and 11Node.js 8.9 - JavaScript / ECMAScript, Python 2.7 and 3, GitApache ANT 1.10.1, Apache Maven 3.5.2, Gradle 4.5, Apache Tomcat 9.0.4, Jenkins Generic WAR 2.89.3 LTS.

E-Framework solutions for DAD lectures and labs were based till year 2017 on VMWare client virtual machines, and they have been provided by IT&C Security Master program - http://ism.ase.ro 

The OLD web address for download will be available for: 

- Linux Ubuntu 12 LTS (Can be updated to Ubuntu 14 with Java 8): with JDK 6 + Eclipse Indigo JEE + Netbeans + Apache Tomcat + JBOSS + GCC - Download 
- Alternatively, the latest versions of software may be installed by students in Ubuntu 14 VM (Oracle VM Box): 
JDK 8 + Eclipse LUNA + GCC - Download U14

Old Sakai instances: http://37.120.250.22:8088/portal https://86.55.177.110:7443 

Evaluation & Gradebook Mark Solution 

The final exam mark is 70%, and the assesments/homeworks/reports/test&quizzes done by the student, during the semester are 30% from the evaluation mark.
All the content related with the tests, assesments and the final exam will be uploaded accordinglly with the established deadlines in the E-Learning System Solution based on SAKAI and provided by IT&C Security Master program - http://ism.ase.ro. The SAKAI e-Learning System Solution will be available soon, the username will be surname.firstname and the password has been communicated during the first lectures.

Assignment 01.

Deadline - YYYY/MM/dd HH:mm: Current Year/03/07 23:50 GMT+2
Problem description: "Create multiple processes for communicating one each other, through UPD multicast/broadcast packages, in order to find out the best mark in the room. Each process has its own mark from 1 to 10 stored in command line for launching or it is stored in a file on the disk. The problem and the solutions have been explained in the lecture 01. You may develop the solution for this problem in Java/Kotlin or C++ / C# / node.js / Swift / Python." 
Upload and packiging: "The ZIP file has the source code files + *.sh/*.bat file for compiling and running the sample. The ZIP filename rule will have DAD_A01_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 02.

Deadline - YYYY/MM/dd HH:mm: Current Year/03/14 23:50 GMT+2 
Problem description: "Build SNMP client program which is able to send SNMP traps in Java over HTTP or Websocket."

Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the PDF file that contains the steps + *.sh file for compiling and running the sample + *.java files with the source code. The ZIP filename rule will have DAD_A02_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 03.

Deadline - YYYY/MM/dd HH:mm: Current Year/03/14 23:50 GMT+2
Problem description: "Build client-server programs in Java. The client read the arrays from files and send them using Sockets and it transmits TCP packets. The servers receives the array and process them using multi-threading. The results are sent back asynchronous by the server to the clients by using Java NIO API."

Please see the discussion thread @ https://stackoverflow.com/questions/56747881/how-to-use-multiple-cores-in-the-java-non-blocking-i-o-nio-api
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the PDF file that contains the steps + *.sh file for compiling and running the sample + *.java files with the source code. The ZIP filename rule will have DAD_A03_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 04.

Deadline - YYYY/MM/dd HH:mm: Current Year/04/15 23:50 GMT+2
Problem description: "Download laboratory/seminary with TCP HTTP Server server - proof of concept, in Linux Ubuntu (download the virtual machine from the current page - http://acs.ase.ro/dad), in order to solve HTTP requests for resources of type *.class. The URL in Internet browser - Mozilla Firefox/Google Chrome for http://ip_server:port_server/myfile.html will display the HTML file if exists, BUT when the browser asks http://ip_server:port_server/MyClassFile.class will display the string chars produced by method 'public String sampleDoGet();'. The one should modify method 'public String processInput(String theInput);' from class 'eu.ase.httpserver.HTTPSeminarProtocol' in order to handle through Java reflection the requiered task."
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the Java source code *.java files + *.sh file for compiling and running the sample. The ZIP filename rule will have DAD_A04_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 05.

Deadline - YYYY/MM/dd HH:mm: Current Year/04/15 23:50 GMT+2
Problem description: "Download lecture and laboratory/seminary in Linux Ubuntu (download the virtual machine from this page - http://acs.ase.ro), in order to solve the following problem:
Develop the JSP and Java bean necessary for Apache Tomcat deploy that will receive via HTTP to vectors and will responde with the sum of the vectors. The Java bean from web will be the RMI client for 3 RMI servers objects, each RMI server object is registred in the same rmiregistry or in different rmiregistry program. Both versions, a) and b) are required.
Version a) - The RMI Server is developed in standard mode as in lecture sample (RMI01 and RMI03 directories).
Version b) - The RMI Server is developed in order to receive the bytecode of business logic (adding two vectors) in the same way as in ComputePi solution - ./downloads_readme/rmidocs/rmiDocsExec"
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the Java source code *.java files + *.sh file for compiling and running the sample. The ZIP filename rule will have DAD_A05_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Assignment 06.

Deadline - YYYY/MM/dd HH:mm: Current Year/04/15 23:50 GMT+2 
Problem description: "Download activity with OpenMPI examples - proof of concept, in Linux Ubuntu (download the virtual machine from the current page - http://acs.ase.ro/dad), in order to highlight 'broadcast', 'gather', 'scatter' on addition operation of two large arrays of integers - e.g. each having 100 000 000 elements. => each array elements occuping in RAM aprox. 400 MB" 
Upload and packiging: "ZIP file uploaded in SAKAI. The ZIP file has the C source code *.c files + *.sh file for compiling and running the sample. The ZIP filename rule will have DAD_A06_SURNAME_FirstName1_FirstName2.zip". The upload is available through SAKAI, please see E-Learning Solution section from this page.

Prerequisites

Fundamentals of Java SE 8 & 11+ plus C/C++ ‘14 + node.js + Networking + Linux/Windows OS | Optional – Python.

Target Group Profile

References

Books

1.  Brendan Burns, "Designing Distributed Systems", Publisher: O'Reilly Media, Inc., Release Date: February 2018, ISBN: 9781491983645; with references to: Kelsey Hightower, Joe Beda, Brendan Burns, "Kubernetes: Up and Running, 2nd Edition", Publisher: O'Reilly Media, Inc., Release Date: October 2019, ISBN: 9781492046530

2. Kalpit Patel, John Carnell, "Spring Microservices in Action", Publisher: Manning Publications, Release Date: July 2017, ISBN: 9781617293986

3. Christina J. Hogan, Strata R. Chalup, Thomas A. Limoncelli, "Practice of Cloud System Administration, The: DevOps and SRE Practices for Web Services, Volume 2", Publisher: Addison-Wesley Professional, Release Date: September 2014, ISBN: 9780133478549 

4. George Coulouris, Jean Dollimore, Tim Kindberg, Gordon Blair, "Distributed Systems: Concepts and Design", Publisher Addison-Wesley; 5 edition (April 27, 2011), ISBN-10: 0132143011, ISBN-13: 978-0132143011 

5. Andrew S. Tanenbaum, Maarten Van Steen, "Distributed Systems: Principles and Paradigms (2nd Edition)", Publisher Prentice Hall; 2 edition (October 12, 2006), ISBN-10: 0132392275, ISBN-13: 978-0132392273

6. My Experience and lectures/labs slides presentations, examples, virtual machines + your visual, kinetic and auditory memory + SAKAI e-Learning System PROVIDED by ISM – IT&C Security Master Program - http://ism.ase.ro | http://acs.ase.ro

*** Additional - offtopic: 

Meher Kasam, Siddha Ganju, Anirudh Koul, "Practical Deep Learning for Cloud, Mobile, and Edge", Publisher: O'Reilly Media, Inc., Release Date: October 2019, ISBN: 9781492034865

Vladimir Silva, "Practical Quantum Computing for Developers: Programming Quantum Rigs in the Cloud using Python, Quantum Assembly Language and IBM QExperience", Publisher: Apress, Release Date: December 2018, ISBN: 9781484242186