COMP0035 Data exploration and preparation

COMP0035 Coursework 01 2024 Coursework specification

1. Table of contents
2. IntroductionThe aim of the combined coursework in this module is for you to select and apply some of the relevantsoftware development and data science techniques that are used in a typical project lifecycle.Coursework 1 focuses on data preparation and database design.Coursework 2 continues from coursework 1, focusing on requirements, application design and testing.This document specifies coursework 1 which is worth 40% of theassessment marks available for themodule. This is an individual coursework.You will submit a written report; and a repository of code files that combined meet the requirementsdetailed in this specification.Aim to make progress each week of first ive weeks of the module, in line with module’s teaching activities.

3. Coursework specification

3.1. Getting started

1. Select a dataset using the ‘group’ selection task in Moodle Week 1 (https://moodle.ucl.ac.uk/mod/choicegroup/view.php?id=6089982). Each ‘group’ option is associated with a data set. ‘Groupselection’ is a Moodle term for the type of task, the coursework is individual.

1. Accept a GitHub classroom assignment. This creates the repository. Instructions are also given inTutorial 1.
2. Login to GitHub.com.
3. Click on the GitHub classroom link (https://classroom.github.com/a/zqVIaThf)
4. Accept the assignment.
5. If prompted, accept to join the comp0035-ucl organisation.Page 1 of 103. Download the dataset for your group choice and add it to your repository. Use the links in Moodle

3.2. General requirements and constraints

• Compile all written work into a single report in either PDF or Markdown format. Name thedocument coursework1.

• The report supports the code and techniques used in the coursework. It is not an essay, be succinct.There are no word limits.

• Demonstrate regular use of source code control using GitHub. Create the repository using theGitHub classroom assignment. Keep the repository private. Keep the repository in the uclcomp0035 organisation.

• You must use the data set allocated to you on Moodle.
• This is an individual coursework. Do not collude with other students using the same data set.
• Use of code AI tools is permitted when writing code. UCL recommends using Microsoft Copilot(https://liveuclac.sharepoint.com/sites/Office365/SitePages/Bing-Enterprise-Chat.aspx) using yourUCL credentials. This must be stated in the ‘References’ section.

• Use relevant techniques from the course, or from data science and/or software engineeringprocesses. Provide references for techniques not included in the course material.

• Diagrams can be hand-drawn and scanned. Using software to draw them does not increase marks.

3.3. Section 1: Data exploration and preparation

The purpose of this section is:

• to use python pandas to describe the data set structure and content; and as a result demonstratethat you understand the data set.

• to use python pandas prepare the data for later use in developing applications. The data youprepare will be used in COMP0034 coursework to create charts in a dashboard app.

• to demonstrate that you can write code that is reusable and understood by other developers.
• to demonstrate that you can apply relevant software engineering and data science techniques.Code quality is also assessed.Use only Python and pandas. matplotlib may be used where pandas DataFrame.plot() (https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.plot.html) is not sufficient.Create charts where they support your exploration and preparation; but do not focus on the visual

aesthetic as this is not assessed.You may need to prepare the data in order to complete the exploration and hence your code may notneatly split between 1.1 and 1.2. Thisis OK, the code structure does not need to exactly match the reportstructure.

3.3.1. Section 1.1 Data exploration

1. Code: Write python code to explore and describe the data structure and content. Including, butnot limited to, size, attributes and their data types, statistics, distribution of the data, etc.Consider potential data quality issues.

2. Report: Describe the results of your exploration of the data. Do not include the code in the report.

3.3.2. Section 1.2 Data preparation

1. Report: Briefly describe a target audience and state at least 3 questions that they might beinterested to explore using the data. This defines the purpose for which you will prepare the data.Page 2 of 102. Code: Write python code to prepare the data such that it can be used to try to answer thequestions for the audience described in step 1. Aim to have sufficient data, and avoid unnecessarydata. The prepared data should be in a format that can be read into one or more pandasdataframes from a file (.csv or .xlsx). If relevant, address any data quality issues dentified insection 1.1.

3. Report: Explain how you ensured the data is relevant for the purpose.
4. Include the original and prepared versions of your data set files in your repository.

3.4. Section 2: Database design and creation The purpose of this section is:

• to demonstrate that you understand the structure of a relational database and the principles ofnormalisation by designing an appropriate database and drawing this as an entity relationshipdiagram (ERD).

• to demonstrate that you can write 代写COMP0035  Data exploration and preparation Python code to create an SQLite database based on the ERD.The database you create can be used in COMP0034 coursework in a data driven web application.

3.4.1. Section 2.1: Database design

Design a relational database that can store the data (based on either the prepared or the raw data set,your choice). Consider normalisation.Document the design as an Entity Relationship Diagram (ERD) that includes the following details as aminimum:

• table(s)
• attributes in each table
• data type of each attribute
• primary key attribute for each table
• foreign key attribute(s) if relevant
• relationship lines between tablesInclude the ERD in your report. An explanation is not required, though you may discuss yournormalisation if relevant.

3.4.2. Section 2.2: Database code Write python code that:

• creates a database structure based on the ERD for an SQLite database file.
• takes the data from the dataset file and saves it to the SQLite database file. Note: do not create adatabase that requires a server such as MySQL or PostgresSQLThe quality of the code is assessedUse relevant Python packages, i.e. pandas and sqlite3.

3.5. Section 3: Tools

The purpose of this section is to demonstrate appropriate and effective use of relevant software engineeringtools.Page 3 of 103.5.1. Section 3.1 Environment management Provide relevant files and instructions that allow the marker to set up and run your code in a Pythonvirtualenvironment. They will use pip and setuptools with thecommands:pip install -r requirements.txtpip install -e .As a minimum, edit the files that were provided in the starter code of the repository:

• requirements.txt: list the packages used in your code
• pyproject.toml: provide basic project details and code package location
• README.md: provide instructions to install and run your code for the data preparation and thedatabase creation

3.5.2. Section 3.2: Source code control Add the URL for your repository to the report.Make regular use of source code control.

3.5.3. Section 3.3: Linting

Use a Python linter to demonstrate how your code meets Python style standards such as PEP8, PEP257.For example:

• state which Python linter you used.
• provide evidence of the results of running the linter.
• if issues are reported by the linter, address these and then run the linter again and show the results.
• if any issue cannot be addressed, explain why not.

3.6. Section 4: References Include code references in comments in the code files close to where it is used.Include all other references, if used, in the report.

3.6.1. Section 4.1 Reference use of AI State either that you used AI, or state that you did not.f you used AI, include the details stated in the UCL guidance (https://library-guides.ucl.ac.uk/referencing-plagiarism/acknowledging-AI#s-lg-box-wrapper-19164308).

3.6.2. Section 4.1 Dataset attribution

Comply with any license condition required for your data set (given in the data set link in Moodle >Resources > Data sets).Each license is different and tells you what has to be cited; e.g. see open government licence v3 (https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/). Typically, but not always,‘attribution’ is required: i.e. include a statement listing who owns the data and its locationPage 4 of 104. Submission Refer to Moodle > Assessment for the deadline date and time.Submit your work on Moodle in the assignment submission. The submission states the upload format:.zip for the code (and report if in markdown) plus .pdf for the report (if not in markdown).GitHub is not an acceptable alternative for submission, though its facility to download the code files aszip may be useful to you.Make sure all files are in the submission. URLs linking to external files cannot be marked as they could bechanged after the submission time. The only exception is where the original data files are too large toupload to Moodle - in this exceptional situation list url(s) to the data files in your report or theREADME.md instead.Do not include your .venv folder in the zip file, this creates unnecessarily large zip files.

Table: Submission checklist

5. Marking

5.1. Module learning outcomes The module’s published learning outcomes that are assessed in this coursework are indicated in the table.Page 5 of 10Learning outcome Coursework 1 ?

1. Describe how software development methodologies can be used to manage thesoftware development process and select and apply an appropriate methodology fora given project.

2. Select and apply techniques for capturing and modelling requirements.
3. Select and apply techniques for modelling an application; and model anapplication using these.Yes -database

4. Select the aspects of a software application can be modelled with the UnifiedModelling Language (UML); and use UML to model different views of anapplication.ERD (notUML)

5. Model the design for a database.Yes

6. Describe testing and recommend an appropriate approach to testing for a givenproject.

7. Recognise the challenges of working in a team and organise themselves and theirgroup to deliver a complex project.Recognise the ethical implications of using data in the context of this course andbe aware of their responsibilities to comply with relevant UCL and UK legislation.Yes

9. Work in a group to apply the skills and knowledge gained in the course to: a)produce a coherent and cohesive specification for an application; and b) select,install, configure and use a set of open-source tools and use these to support thesoftware development cycle for the application.Yes, part (b)The published learning outcomes are being revised. In particular:Data preparation and visualisation are core to the module content yet missing from the publishedlearning outcomes.Following feedback from previous students, the coursework is now individual. Learning outcome 7 isnot addressed; and learning outcome 9 needs to be re-worded.

5.2. Mark allocation

You are expected to spend 18 hours on coursework 1 (45 * 40%).The weighting of each section is shown with an indication of the expected hours of effort required.

5.3. Grading criteria

The coursework is assessed according to the standards set in the standard UCL Computer Science gradingcriteria ( see copy on Moodle in the Assessment section). The criteria most relevant to this assessment are1, 2, 4 and 5.The following tables give the standard UCL CS criteria, and indicators specific to this coursework. Thecoursework is open-ended and allows for different solutions; it is not possible to describe every aspect thatcould be considered.Mostly appropriate use of the expected tools.Low pas40-4Limited, or inappropriate, use of the expected tools.‘Regular’ use of source code control is stated in section 3.3. ‘Regular’ cannot be precisely define sincestudents work over different periods. You are expected to makeprogress on your coursework weekly.Commits over a period of weeks could be considered ‘regular’; commits only during a short period such as1-2 days could not be considered ‘regular’.

6. Appendices

6.1. Code quality This is considered as:Code that is easy for others to read and understand.Code that is re-usable. The focus in this IEP minor is on writing code that could be used in

applications, not simply on whether the code works.When you are writing code consider:code structure, e.g. use of functions, classes, modules, packages.adherence to python conventions (PEP8 style guide (https://peps.python.org/pep-0008/), PEP275docstring conventions (https://peps.python.org/pep-0257/)).

documentation (docstrings, comments).error handling.

6.2. Code that does not fully function

If your code does not fully work, and you cannot ‘debug’ and fix it before submission, then in the relevant

section of the coursework document state as much of the following as you can:What is the code that doesn’t work (e.g. a function name)What you think the problem maybe. This shows you understand the issue even if you cannot solveAny solutions you have tried. This shows that you understand the issue and were able to take stepsto try and resolve it.Clear code documentation (docstrings, comments) is often useful in these situations as the marker canmore easily see what you intended your code to do, even if it does not fully achieve that.Page 10 of 10