Pipeline

Introduction

In programming you may need to apply multiple operation one after the other. Using a pipeline is one way to to this. (See Linux/Unix pipes.)

Any one of the operations could fail. The error needs to be recognized and dealt with.

To see an example of a pipeline of functions (operations) click HERE .

Project #1

Maintain an inventory of product for a store in a CSV file. One item per line.

To see an example of a inventory CSV file click HERE . Use the file for testing.

In this project we are going to create a pipeline of functions. Each function performs a single task on the inventory and passes it to the next function.

The pipeline functions will ...

  1. read the inventory file.
    Report any errors and statistics (i.e counts).

  2. write the inventory data to a file.
    Report any errors and statistics (i.e counts).

  3. verify the inventory data and report errors
    • the id is 6 digits
    • not duplicate ids
    • each entry contains 6 fields
    • there are no empty fields
    • the current count and minimum count are integers
    • the unit price is a float
    Report any errors and statistics (i.e counts).

  4. check the difference in the current and minimum required items
    Report any errors and statistics (i.e counts).

  5. Add items to the inventory.
    Read a file of entries to add to the inventory.
    The format should be the same as inventory format.
    Insert the items in order or sort the inventory at the end.
    Report any errors and statistics (i.e counts).

  6. Delete items from the inventory.
    Read a file of IDs to delete from the inventory. (One ID per line.)
    Report errors and statistics (i.e.counts).

  7. Modify items in the inventory.
    Read a file of IDs, the field to change and a change value.
    Report any errors and statistics (i.e counts).

Getting Started

Use a @dataobject to contain the inventory information.

To see example code to get you started click HERE .

Inventory File Names

Inventory file are prefixed with their creation data (yyyymmdd). (i.e. 20231011_inventory.csv)

Inventory Data Structure

I suggest

Use JOSON data structure?

How to Report Errors

Each function that detects an error should

Normally the bad entries are written to a file. The file entries could corrected and then inserted back into the inventory.

How to Report Shortages

When a shortage detected, print shortages

Normally the shortage information is written to a file.

Verbose

each function should report what is doing and and other statistics (i.e. counts).

Create a pipeline and functions that can perform all of the above one at a time in any order.

This is a very artificial problem, but the purpose is to work with a pipeline of functions. If you would rather do a different pipeline project, talk to the instructor. They are always open to new projects.

Project #2

Recode Project #1. Instead of each function testing for an input of None, use a try ... catch block around the pipeline. This could simplify the work functions and error checking.

Project #3

The system's defined exceptions may not work well for your project. Define your own exception(s).

Project #4

Create a menu to let the user decide the order of operations.

A GUI? Allow the user to drag-and-drop the functions they want and in the order they want? How do you execute code you create inside your program?

Further Study

Note, This can be heavy computer science stuff.

What is a Monad?
What is a Functor?
Monad (functional programming) (Wikipedia)
Functor (Wikipedia)
Functional Programming (Wikipedia)

Note, you don't need to know the computer science to use the concepts.