Welcome to Data analysis with Python - Summer 2019¶
In this course an overview is given of different phases of the data analysis pipeline using Python and its data analysis ecosystem. What is typically done in data analysis? We assume that data is already available, so we only need to download it. After downloading the data it needs to be cleaned to enable further analysis. In the cleaning phase the data is converted to some uniform and consistent format. After which the data can, for instance, be
- combined or divided into smaller chunks
- grouped or sorted,
- condensed into small number of summary statistics
- numerical or string operations can be performed on the data
The point is to manipulate the data into a form that enables discovery of relationships and regularities among the elements of data. Visualization of data often helps to get a better understanding of the data. Another useful tool for data analysis is machine learning, where a mathematical or statistical model is fitted to the data. These models can then be used to make predictions of new data, or can be used to explain or describe the current data.
Python is a popular, easy to learn programming language. It is commonly used in the field of data analysis, because there are very efficient libraries available to process large amounts of data. This so called data analysis stack includes libraries such of NumPy, Pandas, Matplotlib and SciPy that we will familiarize ourselves with during this course.
No previous knowledge of Python is needed as will start with a quick introduction to Python. It is however assumed that you have good programming skills in some language. In addition, linear algebra and probability calculus are prerequisites of this course. The course lasts for seven weeks and gives 5 credit units. It is recommended that you do this course in the end of bachelor degree or in the beginning of masters degree; preferably before the course “Introduction to Data Science”.
Passing the course¶
From the weekly programming exercises you need to get 80% of the points. If you succeed in this, then you can start doing the project work (approximately 1 cu of work). After the project work and its peer reviewing is done, the you can take part in the final exam. The final exam consists of multiple choice questions on the themes discussed in the exercises. The final grade will be based on the project work, its peer review, and the exam.
Programming exercises form the 4 cu massive open online part of the course (MOOC). We provide a certificate for passing this part. After completing the programming exercises, certificates are available from the TMC website, under the user page. Project work, its peer-review and exam are only available for enrolled students. These are required for getting the course marked with a grade and registered as a 5 cu course taken at the University of Helsinki.
Note that this course requires a lot of work! Depending on your background it can take something between 5 to 20 hours per week. In addition to reading the course material, you sometimes may need to consult the online documentation of Python or its various libraries.
Schedule¶
Every week there are 15 - 20 programming exercises that you must solve and then submit to the TMC system for checking. In order to continue with next week’s exercises, you need to get 80% of the points from exercises of the previous week. Note that from some exercises it is possible to get more than one point. These exercises are divided into parts, where the parts are shown as Part 1, Part 2, etc. Each part gives one point. It is possible to submit, and get points for, an exercise you have completed at least partly. You can check your points from the TMC server.
Discussion forum¶
A Telegram chat room for the course has been opened. We recommend that you use the channel either through a web browser or the Telegram desktop application.
You can reach the channel through this link: https://t.me/tkt_dap. The browser version can be reached here Telegram.
The discussion channel is based on peer support. The teachers of the course are participating in the discussion on voluntary basis if time permits.
Please refrain from posting code or stack traces as text or attachments in the chat room. Use tmc paste or another paste site instead and just post the link. Also try to avoid posting complete solutions or spoilers for exercises.
Software libraries used¶
| Library | Documentation |
|---|---|
| numpy | doc |
| pandas | doc |
| matplotlib | doc |
| scikit-learn | doc |
| scipy | doc |
Contents:
- Initializing course environment
- Running Python code
- Frequently asked questions
- When logging to tmc first time, whats the server address it asks?
- TMC login crashes (with mention of tmc-cli.log)
- TMC test fails but all other commands (including submit) work
- Stuff generally doesn’t seem to work in the conda prompt on Windows
- tmc not found
- tmc test says: “Test results: 0/0 tests passed”
- What version of Python should I use? What is the name of the executable?
- How to load a file that resides in the src folder?
- Tests complain about missing attribute assert_called, assert_called_once, …
- ModuleNotFoundError: No module named ‘somelibrary’
- I cannot understand the error message from a failed test case
Week 2:
Week 4:
Week 5:
Week6: