This is an old revision of the document!
Table of Contents
Lab 1 - Scientific Computing
Information, Preparation, Resources, Etc.
Readings to be completed prior to this lab:
- none
Readings to be completed prior to next week's lab:
- none
Goals for this lab:
- Setup your computer for the semester
- Learn how to take screenshots.
- Introduce command line control of your computer.
- Meet the
Terminal
- Put it in your dock
- Customize it
- Learn basic commands and terminology for using the
Terminal
in Mac OS X. - Introduce the MATLAB programming environment.
- Introduce basic MATLAB programs and programming syntax.
Software introduced in this lab
Laboratory Report
Your first lab report will be due one week from tonight.
- Throughout this (and all) lab exercise pages you will find instructions for your lab reports within these boxes.
Housekeeping
Let's make sure the software we'll use most often is easily available by putting shortcuts in our dock (the row of applications at the bottom of your screen). For now, don't worry about knowing what these programs are or what they do.
1. Confirm that you have the icons for MATLAB
and BBEdit
in your dock. Let me know if you can't find one or both.
2. Put a shortcut to the Terminal
app into your dock.
2a. Find the application by going to Applications
–> Utilities
. You should see the Terminal
application.
2b. Drag and drop the icon onto your dock.
A Note on Scientific Computing
Cohen, M. X. (2014). Analyzing Neural Time Series Data: Theory and practice. Cambridge, Massachusetts: MIT Press
Scientific computing is often a frustrating uphill slog punctuated by moments of epiphany and success. It's sort of like fighting a losing war, just hoping to win some battles along the way. This is particularly true for beginners, but can also be true for intermediate and advanced users because, regardless of skill level, we are all generally operating at the edge of our knowledge and pushing the boundaries of what we can achieve. I am sharing this perspective (admittedly, exaggerated for comic effect) of scientific computing because one of the skills I want you to acquire in this course is problem solving. It is tempting to think that problem-solving in science refers to problems with a capital “P”; big ideas, big questions, and big theories. But the fact of the matter is that the daily work of science is most often about problem-solving the “small” practical problems that stand as barriers to achieving our larger goals. The first step in technical problem solving is often the same first step you use to get the phone number of the local Chinese restaurant; Google.
The lab wiki for this course will contain lots of information and many step-by-step tutorials, but you will invariably make mistakes and/or run into problems such as a crashing program or script. I expect you to attempt to solve these issues yourself before coming to me for help.
To that end, here are some tips for self-sufficiency. Also, there are tons of useful OS X Terminal and Unix tutorials online. I've selected a few here.
Part 1: OS X Terminal
EVERY Apple computer running OS X is built on the Unix platform and comes with the Terminal app pre-installed. If you have a Mac, you have Terminal! So you can get familiar with the Terminal on your own machine.
The Mac operating system is a Unix-based graphical user interface. You can therefore interact with your Mac computer using many (most) of the same commands used in Unix/Linux systems. Many of these commands achieve the same things that you can achieve using 'point and click' methods.
We “talk” to the computer by typing instructions on the command-line in the Terminal.app
.
Most things we want the computer to do can be achieved several ways. For example, let's say we want to create a new directory on our desktop.
You're probably familiar with referring to “folders” on your computer; a “directory” is essentially the same thing. When programming it is more common to refer to “directories”. There is a nuanced difference between them, but not one we need to care about!
On our Mac, we can create a new folder on our desktop by clicking on the desktop, selecting File
New Folder
. Alternatively, you could right-click on the desktop and select New Folder
.
But you can also create a new folder from the command-line. Let's give it a shot.
Using Terminal
1. Open Terminal
. You should see a window like the one below. (the colors of yours will look different, but we'll come back to that in a bit)
You can easily copy my code by clicking the little clipboard that appears in the upper-right when you hover over a code box.
2. Type the following command and then press return
cd ~/Desktop
3. Type the following command and then press return
mkdir ./mynewfolder
Voila! You should see a new folder on your Desktop named “mynewfolder”. What just happened?
cd ~/Desktop
told the computer to change your directory to the Desktop (we'll talk about the~/
part in a bit.mkdir
tells the computer to make a directory../
is telling the computer that you want that new folder right here.mynewfolder
is telling the computer what to name the new folder
Why use Terminal??
At this point you are undoubtedly thinking “It was way easier to just use the mouse/trackpad and point-and-click, so why would I ever want to use the command-line!?” Well …
- the command-line gives you direct control over many aspects of your computer that would be hard (or impossible) to access via the graphical interface.
- we can write our commands into a text file (called a “script”) and then tell the computer to read the commands from that file. This is great because it means that we can give the computer lots of step-by-step instructions.
- if we need to run the task 100 times (e.g., we want to create 100 new folders), we can just tell the computer to read the script 100 times. That sounds a lot easier than right-clicking and selecting
New Folder
100 times. - among other great things, this ensures that whatever task we've “scripted” is being done that we're exactly the same way every time we do it. This might be kind of important when we're, say, analyzing data from many subjects.
The majority of what we do in this course will rely on controlling your computer using the command-line (either in Terminal
or Matlab
(a program that will be introduced later in this lab). You should therefore dedicate a sufficient amount of time to getting comfortable with basic Terminal/Unix commands and behaviors. We will spend some time tonight learning a few basic commands. A more comprehensive list of commands can be found here.
Let's get started learning a bit about the Terminal!
Setup your Terminal
Trick it out
If you're so inclined, you can change the way your Terminal
looks. In that same settings window, click on Profiles
to see many different default layouts and color patterns. You can learn more about how to customize it here.
File paths
File paths are simply the directions to a particular file or directory on your computer. You already have an intuitive understanding of these paths, though you might not have ever referred to them as such. Most likely, you think of your file system in terms of “folders” and “sub-folders” arranged in a hierarchy (e.g., my term paper is inside the folder named “psyc class” which is inside the folder “kenyon stuff”). The path is simply directions for how to navigate this folder hierarchy.
When working with the command line it is more common to refer to “directories” and “sub-directories”, than “folders” and “sub-folders”.
1. Open your Mac Finder
by clicking on the finder icon on your dock.
2. Navigate to your Downloads
directory by double clicking on it. You should now see the contents of the Downloads
directory. I assume that this type of interaction with the file-system is very familiar to all of you.
3. To see the filepath for this location select View
–> Show Path Bar
(if you only see an option for Hide Path Bar
then your computer is already set up the way we'd like it to be and you can skip this and continue reading.)
You should now see a list of folders displayed at the bottom of the window. This is the filepath! If we wanted to use the command line to refer to this location we would type /Users/hnl/Downloads
4. Let's try getting to that same location using the command line. Open your Terminal
program by selecting the Terminal icon on the doc.
5. In the terminal window type the following command followed by return
(from now on, pressing return
after each line is always implied).
pwd
pwd
tells the system to “print working directory”. You should see /Users/hnl
, which is your default home directory.
The tilda character ~
is code for you home directory. With regard to paths, whenever your computer sees ~
it will interpret it as /Users/hnl
. Earlier when you entered cd ~/Desktop
into your terminal, you were really telling the computer to go to /Users/hnl/Desktop
6. Before we leave this location, let's see what directories and files are in our current location.
ls
ls
tells the system to list everything in the directory.
7. So we know where we are and what's in this directory, but now we need to get into the Downloads
directory. To do that we need to use the cd
command (change directory). The command below tells our system to change the directory to the Downloads folder using the full path. Go ahead and enter the command below into your terminal window.
cd /Users/hnl/Downloads
8. Finally, make sure that you've successfully navigated to the Downloads
directory by using the pwd
command.
Absolute and Relative Paths
An absolute path is a full set of instructions. It tells the computer how to get someplace starting from the very beginning (i.e., the top of your directory hierarchy). Let's say your parents were coming to campus and planned to meet you in the Crawford Computer lab, but wanted to check out the sculpture in the science quad along the way. You would give them directions from their house to the science quad.
So far, so good. But what directions would you give them to get from the science quad to the computer lab? You could give them directions from their home to the computer lab. This is similar to using an absolute path. You ignore the current location and it's proximity to the desired location and simply give the entire path.
However, I doubt that any of you would actually do this. Instead, you would tell your parents “when you're done checking out the sculpture, come in the back entrance of Sam Mather, come up the stairs, etc.”. In other words, you would tell them how to get to their desired location from their current
location. This is a relative path.
When we are using the command line to navigate our directory hierarchy ..
means 'the directory above'. So what do you think the following code would do?
cd ..
Correct! It would move us up one level in our directory hierarchy. If we were in /Users/hnl/Desktop
it would move us to /Users/hnl
And if we wanted to move up two levels we could use
cd ../..
If we were in /Users/hnl/Desktop
it would move us to /Users
The /
symbol in a path divides directories and/or files.
You can think of it like the comma in: Gambier, OH, United States of America
So now let's think about this in terms of relative paths.
9. Let's first go back to our home directory using what we've learned. We can type
cd /Users/hnl
or simply type
cd
or simply type
cd ~
From the computer's perspective, those three lines of code are all identical!
No matter where you are in the directory hierarchy, typing cd
and pressing return
will bring you back to your home directory of /Users/hnl
, and we know that ~
is shorthand for /Users/hnl
.
10. We know we can get to the Downloads folder from our current location by using the absolute path and typing cd /Users/hnl/Downloads
that we used in Step #7. But let's use a relative path instead.
When using the terminal, anything that comes after a hashtag #
is annotation that is for human eyes only. It is not read by the computer.
You should read (and write) these annotations as they describe what is being done by the code. You should always strive to annotate your code so that 1) it's easily understood by yourself or other humans and 2) so that at a later date you can understand what you were trying to do. I promise you, you will not remember!
NOTE: If you copy and past the entire block of code into your terminal, everything will execute properly but you might also get lots of “command not found” errors. This is because #
annotation is really intended to be used when running scripts rather than single lines of code at the command-line.
If we do not specify a path, the computer will assume we are starting from wherever we currently reside.
Therefore, the following command will work because computer assumes we are starting from our current location, which happens to be /User/hnl
, which is one level up from Downloads
.
# change to the Downloads directory cd Downloads
The following command would have worked exactly the same, but it is more explicit that we are starting from our current location. Whereas ..
refers to the directory above our location, .
refers to our current directory.
# Change to the Downloads directory cd ./Downloads
After typing cd ~
I wrote that either of the two commands above would successfully move us to the Downloads directory. But if we ran both of them, one after the other, what would happen?
cd ~ cd Downloads cd ./Downloads
Oops, it didn't like that. Why not? If it's not clear to you why not, try running each of the three lines of code one at a time, and after you run each line, run pwd
to see where you are.
11. So now we are in /Users/hnl/Downloads
. What command should we use if we want to navigate to /Users/hnl/Documents
. You now know we've got two options.
# Absolute path cd /Users/hnl/Documents # Relative path # The Documents directory is parallel to the Downloads directory in our directory hierarchy. So to get there, we # need to go up one level, then down into Documents. cd ../Documents
On the one hand, it tends to be better to use absolute paths when you're writing scripts or programs because it ensures you're going where you think you're going and makes your code more robust (it should work no matter your starting point). On the other hand, it's better to use relative paths when you're working interactively (as we've been doing) because it spares you from typing out the full path.
File & Directory Control
You've already learned a couple of important terminal commands; cd
, pwd
, and ls
. In this section we'll learn a few more that can be used to create and delete, move, copy, and rename files and directories. And remember you can find a list of the most used commands here.
Over the next series of steps we'll create a new folder and some new files and then play with moving, copying, renaming, and deleting them.
1. To create a new directory we use the mkdir
command (make directory). Let's use that command to create a new directory named “foo” on our Desktop.
mkdir /Users/hnl/Desktop/foo
You should now see a folder named foo on your Mac Desktop. Kinda cool, no?
Terminal has an auto-complete feature that can save you some typing. If you had typed mkdir /Users/hnl/De
and then hit the TAB
key Terminal would have auto-completed the word Desktop
.
However, if you typed mkdir /Users/hnl/D
and then hit the TAB
key Terminal would just beep at you. That's because there is more than one item within /Users/hnl
that begins with D
(Desktop
, Downloads
, Documents
) so Terminal doesn't know which to autocomplete.
2. Delete the folder you just created. To do so we'll use the rmdir
command (remove directory).
rmdir /Users/hnl/Desktop/foo
Voila. Keep in mind that rmdir
will only work if the directory is empty.
Be. Very. VERY. Careful. When you use the command line to remove files and directories they are gone FOREVER. They do not go into a Trash bin from where they can be recovered. They are truly deleted.
3. Now we'll recreate our “foo” directory and also create a sub-directory named “subfoo” inside of “foo”. Try this
mkdir /Users/hnl/Desktop/foo/subfoo
Oops! That doesn't work and we get an error. This is because the “foo”directory does not exist, so the computer is confused about where to put the “subfoo” directory. Does this mean we have to create the “foo” directory and then create the “subfoo” directory? Not at all. We just need to give a little extra info at the command line. Try this
mkdir -p /Users/hnl/Desktop/foo/subfoo
Success! The -p
part of our code is called a flag
. It is an option that modifies the mkdir
command. It tells the computer to create the directory “subfoo” and create any other directories in the path that do not yet exist.
Almost all commands (aka “utilities”) have options that can be accessed using flags. For instance -l
is a flag that changes the output of the ls
command you used earlier. Give it a shot.
4. Now that I think about it, “subfoo” is not a great name. I think I'd prefer to name that directory “babyfoo”. To rename files or directories we use the mv
command (move). Not surprisingly, we also use this command to move files and directories. But here we'll use it to rename our “subfoo” directory.
mv /Users/hnl/Desktop/foo/subfoo /Users/hnl/Desktop/foo/babyfoo
5. Now we'll create some files inside of our directories using the touch
command. The following code will create a couple of text files in the babyfoo directory and a Word document file in the foo directory. After running the code, see for yourself that the files were created. You can show off and use the command line, or just double-click the folder on your Desktop.
Instead of typing out the commands, you can just copy and paste them into your terminal. However, make sure you understand what each line is doing!
touch /Users/hnl/Desktop/foo/babyfoo/file1.txt touch /Users/hnl/Desktop/foo/babyfoo/file2.txt touch /Users/hnl/Desktop/foo/file3.doc
6. We've already seen mv
used to rename a directory. Here we'll use it to move the file3.doc from the foo directory to the babyfoo directory.
mv /Users/hnl/Desktop/foo/file3.doc /Users/hnl/Desktop/foo/babyfoo/file3.doc
7. Now all three files are in the babyfoo directory. D'oh! I actually need to have file3.doc in both directories. So let's use the cp
command (copy) to make a copy of the file in the foo directory.
cp /Users/hnl/Desktop/foo/babyfoo/file3.doc /Users/hnl/Desktop/foo/file3.doc
8. Actually, it'd be best if there were copies of all three files in both directories. We could use the cp
command to copy each file, but let's be a little more efficient. We'll use the wildcard symbol *
to make a copy of all files that end with .txt
.
cp /Users/hnl/Desktop/foo/babyfoo/*.txt /Users/hnl/Desktop/foo
9. Okay okay okay, I now realize that I don't need the file3.doc file in the foo directory after all. We've already seen that rmdir
will remove (i.e. delete) a directory, but here we'll use rm
to remove a file.
rm /Users/hnl/Desktop/foo/file3.doc
Scripting
So far we've used the command line interactively. That is, we wrote a command, the computer carried out our command, we wrote another command, the computer carried out that command, and so on. This, however, is not the only way to have the computer carry out our marching orders. We can instead write out all of the things that we want the computer to do and save them into a shell script.
There are lots of different shells (e.g. sh
, tcsh
, zsh
), but they all pretty much do the same thing. They mainly differ in style and syntax. They're not really “languages” in the programming sense of the word, but it's okay to think of them as such.
zsh
is the default shell used in OS X, but historically I've written in bash
. Inside of your script you can tell the shell to use any shell language that you want. That is, to run bash
code in a zsh
shell you would simply put the following code as the very first line in your script #!/bin/bash
.
I am hoping to update my code to zsh
but you might see some scripts using one and other scripts using the other.
In the last section we created, renamed, moved, copied, and deleted some files and directories in a long series of interactive steps. The script below would accomplish the same thing without us needing to write each command interactively. Instead the computer would simply read our list of commands and carry out each one in order.
1. Delete the “foo” directory from your Desktop (you can test your command line skills or just drag and drop it into the trash).
2. Download the following script to your computer by clicking on the tab that reads “myFirstScript.sh”.
- myFirstScript.sh
#!/bin/zsh # create the foo directory mkdir /Users/engella/Desktop/foo # remove the directory we just created rmdir /Users/engella/Desktop/foo # Do not run this next line of code # mkdir /Users/engella/Desktop/foo/subfoo # make the foo directory and subfoo sub-directory mkdir -p /Users/engella/Desktop/foo/subfoo # rename subfoo to babyfoo mv /Users/engella/Desktop/foo/subfoo /Users/engella/Desktop/foo/babyfoo # create two text files in the babyfoo directory touch /Users/engella/Desktop/foo/babyfoo/file1.txt touch /Users/engella/Desktop/foo/babyfoo/file2.txt # create on Word doc in the foo directory touch /Users/engella/Desktop/foo/file3.doc # move the Word doc from the foo to the babyfoo directory mv /Users/engella/Desktop/foo/file3.doc /Users/engella/Desktop/foo/babyfoo/file3.doc # copy the Word doc in the babyfoo directory into the foo directory cp /Users/engella/Desktop/foo/babyfoo/file3.doc /Users/engella/Desktop/foo/file3.doc # delete the Word doc from the foo directory rm /Users/engella/Desktop/foo/file3.doc
3. Navigate to your Downloads
directory in your Terminal and run the script using the following command.
source ./myFirstScript.sh
This should have carried out all of the steps we previously did interactively. Obviously, there's not much value in this series of silly steps. But scripting can be an extremely powerful tool that makes repeating tasks easy, reliable, and leaves a document of what has been done.
You can think of source
as us telling the computer to run the script. There are fancier ways to do this, but that's for another time. (If you simply must know, start here)
What's the difference between scripting and programming?
The line can get kinda blurry, but basically scripting tells the computer what to do, whereas programming tells the computer how to do it. Almost everything we do in this course will be on the scripting side of things.
Disclaimer: I am not a programmer! I know just enough scripting and programming to get by (and just enough to get myself in trouble). Your local computer scientist might have much more to say about this difference.
Editing Shell Scripts
You should use BBEdit whenever you write or edit shell scripts. NEVER use word processing software (e.g., MS Word). Word processing software is terrible for writing scripts because it embeds all sorts of hidden text and symbols that will confuse our shell. There are many good text editors out there, but we'll use BBEdit because it's free and easy.
To make things easier, let's make sure that your computer knows to use BBEdit when opening files with a .sh extension.
- right-click on the file and select
Get Info
- In the dropdown menu under
Open with:
selectBBEdit
- Press the
Change All…
button
Now whenever you double click on an .sh
file it will automatically open in BBEdit
LAB REPORT Part 1 - #1
LAB REPORT Part 1 - #1
- Write a script that will carry out the following steps.
- Annotate each line of your code with a description of what that line will accomplish (you can simply use each of the instructions below as your annotations). You can use the example script above as reference.
- Use an absolute path to navigate into your
Movies
Directory - Print (i.e., display on the command line) your current working directory
- Use a relative path to navigate into your
Documents
directory - Create a text file named
lab1.txt
- Use a relative path to navigate back to your home directory
- From your home directory, list the contents of your
Documents
directory - Use an absolute path to navigate back to your “Documents” directory
- Delete the lab1.txt file (be careful to only delete the
lab1.txt
file….remember that there is no way to recover accidental deletions
- Hint: Don't forget to put the code that tells the computer to use the zsh shell at the top of your script.
- Save the file as
YOURLASTNAME_lab1.sh
. Note that our bash scripts should have a.sh
file extension.
Part 2: MATLAB
MATLAB is a multi-platform (that means it runs on Linux/Mac/PC) programming language that is designed to excel at numerical computing, particularly numerical matrix manipulation (indeed, it's name means matrix laboratory). It is therefore the preferred language for many scientific, engineering, and economics applications. Many of the EEG and fMRI programs and applications that we will use throughout the semester are written in MATLAB.
Kenyon has a MATLAB license, which means you can install it on your personal computer for free!
MATLAB Introduction
To open MATLAB click on the icon in your dock.
You will need to authorize MATLAB using your Kenyon email and credentials.
- The first pop-up window requires your full Kenyon email address.
- The second pop-up requires you to put in your Kenyon username and password.
The window that opens will look like the one below (without the shaded boxes). Note that there are several sub-windows.
- The PURPLE shaded area shows you the path to your current directory.
- The RED shaded area is your command window. This is similar to the command line that you used in Terminal in the previous section.
- The GREEN shaded area is your workspace.
Some of the path navigation commands you learned in the Terminal will also work in the MATLAB command window. Try using cd
, pwd
, and ls
. Other file management commands will not work. For instance, if you wanted to create a text file named “testfile.txt” and typed touch testfile.txt
in the MATALAB command window you would get an error (whereas typing this in the terminal would successfully create the file.)
We can have MATLAB run shell commands by putting !
at the start of our command. This tells MATLAB to run the command as the Terminal would. MATLAB essentially becomes nothing more than a Terminal window for that one command. This is convenient for handling small file management jobs without having to switch programs.
% The code below will not work because the Terminal file management commands don't work in MATLAB touch /Users/hnl/Desktop/testfile.txt % The code below will work because putting the ! at the start of the command tells MATLAB % to use the bash shell (i.e. Terminal) to carry out the command rather than MATLAB ! touch /Users/hnl/Desktop/testfile.txt
We learned that putting #
at the start of a line in bash script makes that line 'invisible' to the computer and is a convenient way to annotate your code. There is a similar feature in MATLAB, but the comment symbol is %
# This text would be invisible to the shell, but not Matlab.
% This text would be invisible to Matlab, but not to the shell.
Using MATLAB
As with Unix/zsh/OSX Terminal, there are many(!) MATLAB tutorials available online. Let's take a look at one offered by MathWorks (the company that makes MATLAB)
1. Go to this MATLAB page.
2. Select the MATLAB Onramp class. (you'll probably need to enter your login credentials again)
3. Complete the Commands and MATLAB Desktop and Editor modules.
- These two are the only modules you are required to complete, but I strongly recommend that you continue to use this, or other resources, to strengthen your MATLAB skills.
4. When you've completed both modules take a screenshot of your progress.
- click the hamburger icon (blue box with three blue stripes) in the upper left corner.
- capture the entire screen or, at least, the portion that shows your progress and your name. (see example below)
- If you've already closed the window, you can just log back into the course to access your progress.
Taking Screenshots
You will need to take many screenshots throughout the semester. You should spend a few minutes learning the various keyboard shortcuts, but I've included the most essential below. By default, screenshots will be saved to your desktop,
shift
command
3
- Capture the entire screen
shift
command
4
- Capture a portion of the screen
Tip: To ensure the highest quality images, it's generally better to select the relevant portion of your screen, rather than capturing the entire screen and cropping it down.
I would like you to watch a couple of 'Introduction to Matlab' videos before our next class. I also strongly recommend that you spend some time going through some online tutorials on your own time. For the most part though, I expect that you will learn MATLAB as we use it.
MATLAB Intro Vids
Please watch the following videos.
LAB REPORT Part 2
LAB REPORT Part 2 - #1
- There are no lab report questions for this part of the lab.