Difference between revisions of "Workflow"

From CSE330 Wiki
Jump to navigationJump to search
 
(59 intermediate revisions by 7 users not shown)
Line 7: Line 7:
 
=== Running Commands ===
 
=== Running Commands ===
  
−
''Mac OS X:'' You will be running commands on your local machine using '''Terminal''', which you can find in your Applications -> Utilities folder.  You will be connecting to your server via SSH also in Terminal.
+
''Mac OS X:'' You will be running commands on your local machine using '''Terminal''', which you can find in your Applications -> Utilities folder.  You will be connecting to your server via SSH.
  
−
''Windows:'' You will be running commands on your local machine using '''Cygwin''', which you will install in the next section.  You will be connecting to your server via SSH also in Cygwin.
+
''Windows:'' You will be running commands on your local machine using '''WSL''' or '''Cygwin''', which you will install in the next section.  You will be connecting to your server via SSH.
  
 
=== Version Control ===
 
=== Version Control ===
  
−
You will be using '''SourceTree''' as a frontend to the '''Git''' version control system.  Git enables you to keep track of changes to your source code, and also push your changes up to the CEC.
+
You will be using '''Git''' as your version control system.  Git enables you to keep track of changes to your source code.  If you're using VS Code, the Git integration makes is very user friendly.
  
 
=== Text Editing ===
 
=== Text Editing ===
  
−
You will be using '''Komodo Edit''' as your text editing tool. Komodo Edit will also enable you to upload files to your Amazon cloud instance.
+
Although we highly recommend '''VS Code''', you may use '''Sublime Text''', '''Komodo Edit''', '''Notepad++''' (for Windows), and '''TextWrangler''' (for Mac). You can also use an IDE, like '''PHPStorm''', if you prefer.
 +
 
 +
=== File Transfer ===
 +
 
 +
In order to deploy your projects to your EC2 instance, you need some sort of method to transfer files from your local machine to your instance using SFTP. One option is using '''FileZilla''', an FTP client, to transport files from your local computer to your server.
  
 
=== Flowchart ===
 
=== Flowchart ===
Line 25: Line 29:
 
[[File:CSE-330-Workflow.png]]
 
[[File:CSE-330-Workflow.png]]
  
−
The flowchart illustrates 3 methods to copy files to your cloud instance: via your text editor, via an external SFTP client, and by cloning your remote Git repository.  We will be primarily using Option 1 in CSE 330 this semester.
+
The flowchart illustrates 3 methods to copy files to your cloud instance: via your text editor, via an external SFTP client, and by cloning your remote Git repository.  We will be primarily using '''Option 2''' in CSE 330 this semester.
 +
 
 +
In the flowchart, your ''Git Client'' is SourceTree and your ''Text Editor'' is Komodo.  The ''SFTP Client'' could be FileZilla.
  
−
In the flowchart, your ''Git Client'' is SourceTree and your ''Text Editor'' is Komodo.  The optional ''SFTP Client'' could be FileZilla.
+
== Installing a Terminal Emulator ==
 +
{{RequiredInstructions|content=
  
−
== Cygwin ==
+
Windows is based on a fundamentally different architecture than Linux, meaning that, if you want to run Linux programs on Windows, you'll have to enable or install a special terminal emulator for it. Mac users do not need to install a terminal emulator since MacOs itself is a derivative of Unix, which is an ancestor of Linux.  The Terminal utility is all you need.
 +
 
 +
=== Windows Subsystem for Linux ===
 +
 
 +
Starting in Windows 10, Windows has the ability to emulate Linux, meaning that you can run Linux commands on Windows, without having to install Cygwin. To enable WSL, follow [https://msdn.microsoft.com/en-us/commandline/wsl/install_guide these instructions]. Once those instructions are complete, you can open a shell by clicking on the Start menu, typing "Bash", then clicking on the entry labeled "Bash on Ubuntu on Windows".
 +
 
 +
=== Cygwin ===
  
 
'''Cygwin''' is a Linux Terminal emulator for Windows.
 
'''Cygwin''' is a Linux Terminal emulator for Windows.
  
−
Mac OS X users do not need to install Cygwin since OS X itself is a derivative of Unix, which is an ancestor of Linux.  The Terminal utility is all you need.
+
==== Installation ====
−
 
 
−
=== Installation ===
 
  
 
The CEC lab machines already have Cygwin installed.  Follow these instructions to get Cygwin up and running on your personal PC.
 
The CEC lab machines already have Cygwin installed.  Follow these instructions to get Cygwin up and running on your personal PC.
Line 44: Line 55:
 
#* Net -> openssh
 
#* Net -> openssh
 
#* Net -> openssl
 
#* Net -> openssl
−
#* Python -> python (you will need this for Module 4)
+
#* Python -> python3 (you will need this for Module 4)
 +
#* Devel -> git (version control system)
 
#* One or more of the following:
 
#* One or more of the following:
 
#** Editors -> vim
 
#** Editors -> vim
Line 51: Line 63:
 
# Allow the installation to complete.
 
# Allow the installation to complete.
  
−
Throughout the rest of this course, when we say to "open a terminal", if you are on Windows, we mean to "open Cygwin".
+
Throughout the rest of this course, when we say to "open a terminal", if you are on Windows, we mean to "open WSL/Cygwin".
−
 
 
−
== SSH Client Configuration ==
 
−
 
 
−
Secure SHell, or '''SSH''', is the leading interface for connecting from your local computer to something on the cloud.  You will be using exclusively SSH to execute commands and upload files to your cloud instance.  If you use a Git hosting service like BitBucket or Github, you will need SSH in order to save your changes there as well.
 
−
 
 
−
This section discusses how to set up your computer for SSH.
 
−
 
 
−
=== SSH Keys ===
 
−
 
 
−
SSH can work using password-based authentication, but it is more common nowadays, and also more secure, to use ''public and private keys'', also known as a ''key pair''.
 
−
 
 
−
Here's how it works.  You have a private copy of your key, called your '''private key'''.  Servers to which you want to connect have an analog to your private key called your '''public key'''.  When you attempt to connect to a server using your private key, the server checks to see if any of its public keys "fits" your private key, and if it finds a match, it lets you in.
 
−
 
 
−
You can think of the ''private key'' as the "key" and the ''public key'' as the "lock".
 
−
 
 
−
#: '''Note:''' Once you have downloaded your key, set the permissions on it such that only you can read it; that is, the *.pem file you downloaded should have permissions <code>-r--------</code> on your computer.  For more information on setting file permissions, refer to [[Linux#File Permissions|the Linux guide]].  (Can you think of any reason why you wouldn't want other people viewing your private key file?)
 
−
 
 
−
=== Creating an SSH Key Pair ===
 
  
−
You should create your own SSH Key Pair.  You will keep the private key on your own computer, and you will upload the public key to all places to which you need to connect.
+
}}
  
−
To generate a key pair, open Terminal or Cygwin and run the following command:
+
== Github ==
  
−
<source lang="bash">$ ssh-keygen</source>
+
{{RequiredInstructions|content=
  
−
Keep the default location.  For the purposes of this course, you do not need to enter a passphrase, although you would probably want one in real life.
+
'''Github''' is a service that enables you to store your code for free in the cloud.
  
−
Your keys should have been created in ~/.ssh.  Check to make sure they exist using the ''ls'' command:
+
=== Creating a Github Account ===
  
−
<source lang="bash">
+
Go to [https://github.org/ Github.org] to create an account.
−
$ ls ~/.ssh
 
−
id_rsa  id_rsa.pub
 
−
</source>
 
  
−
''id_rsa'' is your private key, and ''id_rsa.pub'' is your public key.
+
'''Important:''' Use your @wustl.edu e-mail address to create your account. This will put your account on the "educational plan", which removes some limitations of the free account.
  
−
''' Cygwin Only:''' In order to make the SSH agent run when you start Cygwin, you need to add a couple of lines to your .bashrc file:
 
  
−
<source lang="bash">$ echo "eval `ssh-agent -s`; ssh-add" >> ~/.bashrc # Cygwin only
+
=== Adding Your Key to Github ===
−
$ source ~/.bashrc # reload the Bash shell</source>
+
'''Not uploading your key is perfectly fine, and in many cases easier. Here are the instructions though, if you'd like'''
  
−
=== Connecting to your Cloud Instance ===
+
# First make sure that you've finished the [[SSH]] guide configuration instructions.
 +
#: We will use the same key for connecting to your remote server as for connecting to Github.  (This is completely valid practice.)
 +
# Go to the "SSH keys" section of the settings panel (Manage Account -> SSH Keys)
 +
# Add a new key.
 +
#* I recommend naming it after the computer you're using; if you're in the CEC, for example, you could name it "Warehouse Cygwin".
 +
#* The contents of your key is the contents of your ''id_rsa.pub''.  It looks like <code>ssh-rsa AAAAB3blahblahblahblah yourname@somedomain</code>.  It is exactly the same thing that you had to paste into the ''authorized_keys'' file on your EC2 instance.
  
−
Now that you have your own public/private key pair, we are going to give the public key to your EC2 instance.
+
You will now be able to push code to your Github repository using your personal SSH key.
  
−
In order to access your instance, we need to use the *.pem file you downloaded in the [[Amazon Web Services]] guide.  This file is nothing more than a private key.
+
}}
  
−
<span style="color:red">QUESTION: Should students use the *.pem key pair throughout the process, or should they give their public key to Amazon?  (Discuss.)</span>
+
== FileZilla ==
  
−
#: '''Note:''' Once you have downloaded your key, set the permissions on it such that only you can read it; that is, the *.pem file you downloaded should have permissions <code>-r--------</code> on your computer.  For more information on setting file permissions, refer to [[Linux#File Permissions|the Linux guide]].  (Can you think of any reason why you wouldn't want other people viewing your private key file?)
+
{{RequiredInstructions|content=
−
 
 
−
=== SSH Client Configuration ===
 
−
 
 
−
==== Unix-Based Systems (including Mac OS X) ====
 
−
 
 
−
Mac OS X is based on BSD, a flavor of Unix.  As such, Mac OS X comes pre-built with all the tools you need to use SSH!  Simply fire up Terminal and enter the command
 
−
 
 
−
ssh username@hostname
 
−
 
 
−
To use SSH with a key pair, use the command
 
−
 
 
−
ssh -i /path/to/key.pem username@hostname
 
−
 
 
−
==== Non-Unix-Based Systems (including Microsoft Windows) ====
 
−
 
 
−
Unfortunately, using SSH with Windows is more complicated. Amazon provides a great tutorial on how to connect to a virtual machine from Windows (follow this link if you are in the Urbauer Lab) [http://docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide/ConnectToInstanceLinux.html#connect-from-windows-machine]. It is necessary to install an SSH client to support the connections.  A widely used SSH client for Windows is PuTTY.  You can download PuTTY from http://www.chiark.greenend.org.uk/~sgtatham/putty/
 
−
 
 
−
PuTTY is fairly simple and straight forward with one caveat: Amazon's .pem key pair files are not compatible with PuTTY keys. In order to convert ''.pem'' keys to a PuTTY ''.ppk'' privte key file, you should use the puttygen.exe utility available from the same page [http://www.chiark.greenend.org.uk/~sgtatham/putty/] as PuTTY.
 
−
Next select import under the conversions menu,load the amazon ''.pem'' key file and press the save private key button. Be sure to save the file in the directory where PuTTY looks for its keys.
 
−
 
 
−
Copy and paste works similarly to the X Window System in Unix. You use the left mouse button to select text in the PuTTY window. The act of selection automatically copies the text to the clipboard: there is no need to press Ctrl-Ins or Ctrl-C or anything else. In fact, pressing Ctrl-C will send a Ctrl-C character to the other end of your connection (just like it does the rest of the time), which may have unpleasant effects. The only thing you need to do, to copy text to the clipboard, is to select it.
 
−
 
 
−
To paste the clipboard contents into a PuTTY window, by default you click the right mouse button. If you have a three-button mouse and are used to X applications, you can configure pasting to be done by the middle button instead, but this is not the default because most Windows users don't have a middle button at all.
 
−
 
 
−
Also, here is a good PuTTY tutorial that you might find useful to get started: http://kb.mediatemple.net/questions/1595/Using+SSH+in+Putty+%28Windows%29
 
−
 
 
−
=== SSHFS ===
 
−
 
 
−
SSHFS is a filesystem client which allows secure mounting of remote file systems. While there are other ways to mount remote file systems, SSHFS has the advantage of being able to mount a file system located on any host that has an SSH daemon running without any host side installation or configuration. This means that you can easily access and edit your files using all of your local applications including IDEs.
 
−
 
 
−
As you may have inferred from the name, the underlying implementation utilizes SSH File Transfer Protocol in combination with FUSE, a package now included in the kernel that allows unprivileged users to easily create their own file systems in userspace (see the wikipedia entry for more information [http://en.wikipedia.org/wiki/Filesystem_in_Userspace]).
 
−
 
 
−
To mount a share using password based authentication, the command is
 
−
sshfs user@domain:/path/to/remote/directory /path/to/local/mountpoint
 
−
e.g. To mount the directory /home/joe/myfiles in the user ''joe'''s home directory for a machine with the domain schmoesfiles.org using SSHFS you would enter the command
 
−
sshfs joe@www.schmoesfiles.org:myfiles
 
−
 
 
−
Note that if you are using public key authentication, the command to mount the remote share is slightly different
 
−
sshfs -o IdentityFile=/path/to/private/key user@domain:/path/to/remote/directory /path/to/local/mountpoint
 
−
 
 
−
To unmount the filesystem you can use the following command
 
−
fusermount -u /path/to/local/mountpoint
 
−
 
 
−
=== SFTP ===
 
−
 
 
−
Any server running an SSH server is also compatible with '''SFTP''' or Secure File Transfer Protocol.  (Compare to FTP, or File Transfer Protocol.)  SFTP is a convenient way to edit files on your computer and then upload them to your server in just a few clicks.
 
  
 
You can use SFTP from the command line, or you can use any GUI file transfer client.  All FTP clients I have seen also support SFTP.  One popular FTP client is [http://www.filezilla-project.org/ Filezilla].
 
You can use SFTP from the command line, or you can use any GUI file transfer client.  All FTP clients I have seen also support SFTP.  One popular FTP client is [http://www.filezilla-project.org/ Filezilla].
−
 
−
==== Using FileZilla ====
 
  
 
# Download and install FileZilla from http://filezilla-project.org/download.php?type=client
 
# Download and install FileZilla from http://filezilla-project.org/download.php?type=client
 
#: If you are using a lab computer, you can save it to your H drive.
 
#: If you are using a lab computer, you can save it to your H drive.
−
# When you launch FileZilla, go to Edit→Settings or FileZilla→Preferences, and go to the SFTP options (under Connection).  Click "Add keyfile…", and choose the *.ppk file that you made earlier using Puttygen.  (If you lost that file, you can choose your *.pem file, and FileZilla will convert it for you.)
+
# When you launch FileZilla, go to Edit→Settings or FileZilla→Preferences, and go to the SFTP options (under Connection).  Click "Add keyfile…".
 +
#* Choose your ''id_rsa'' file.  '''OS X Tip:''' Press Shift+Command+Period to reveal hidden files in the file chooser window.
 +
#* FileZilla will want to make a *.ppk file.  A good place to save it would be the .ssh directory as ''id_rsa.ppk''.
 
#: When finished, press '''OK''' (not the red ×) to save your changes.
 
#: When finished, press '''OK''' (not the red ×) to save your changes.
 
# Back on the main FileZilla screen, there are four fields: Host, Username, Password, and Port.  Fill them in as follows:
 
# Back on the main FileZilla screen, there are four fields: Host, Username, Password, and Port.  Fill them in as follows:
−
#* '''Hostname:''' sftp://ec2-xxx-xx-xx-xxx.compute-1.amazonaws.com/
+
#* '''Hostname:''' sftp://<ec2-xxx-xx-xx-xxx.compute-1.amazonaws.com/>
 
#* '''Username:''' The username you created on your server.
 
#* '''Username:''' The username you created on your server.
 
#* '''Password:''' Since you are using a key, you may leave this blank.
 
#* '''Password:''' Since you are using a key, you may leave this blank.
Line 164: Line 113:
 
#* '''Port:''' 22
 
#* '''Port:''' 22
 
# Then click Quickconnect.  If everything is configured correctly, FileZilla should log into your server.
 
# Then click Quickconnect.  If everything is configured correctly, FileZilla should log into your server.
−
 
  
 
[[File:FileZilla.png]]
 
[[File:FileZilla.png]]
−
 
  
 
On the left and right of the FileZilla window, you can drag files between your computer and your server.  Thus, you can edit a file in a text editor like Notepad++ on your decktop, and then upload it to your server by simply dragging it from the left pane to the right pane in FileZilla.
 
On the left and right of the FileZilla window, you can drag files between your computer and your server.  Thus, you can edit a file in a text editor like Notepad++ on your decktop, and then upload it to your server by simply dragging it from the left pane to the right pane in FileZilla.
  
−
== SourceTree ==
+
}}
−
 
 
−
'''SourceTree''' is a tool that enables you to perform version control using a graphical user interface.  It is free and available for Windows 7+ and Mac OS X 10.6+.
 
−
 
 
−
SourceTree is available on the Mac App Store.  Windows users (and OS X users who don't like the App Store) may install it from http://sourcetreeapp.com/
 
−
 
 
−
=== Git ===
 
−
 
 
−
'''[[wikipedia:Git (software)|Git]]''' is a ''distributed version control system'' that enables you to track changes to your code and work with collaborators on the same code base.  In CSE 132, you used ''Subversion (SVN)'', which is also version control system.
 
−
 
 
−
Git, like SVN, is a command-line tool.  However, SourceTree enables you to use Git without having to pull up a Terminal window every time you need to perform an action.
 
−
 
 
−
In Module 5, you will learn more advanced techniques for using Git.  However, for Modules 2-4, you will be using Git primarily as a method to push your changes to your remote repository.
 
−
 
 
−
=== Configuring Git/SourceTree ===
 
−
 
 
−
Once you install SourceTree, you need to tell it your name so that it knows how to tag changes you make.
 
−
 
 
−
# Open the SourceTree Preferences (SourceTree->Preferences on OS X or Tools->Options on Windows).
 
−
# Check the box that says "Allow SourceTree to modify your global Mercurial and Git configuration files".
 
−
# Enter your name and e-mail address.
 
−
#: '''Note:''' This actually does the same thing as running <code>git config --global user.name <NAME HERE></code> and <code>git config --global user.email <EMAIL HERE></code> from the command line.
 
−
# Finally, go to the "Git" tab in the preferences and uncheck "Use the staging area".
 
−
#: You will learn about staging in Git in Module 5, but for Modules 2-4 we will skip the staging area.
 
−
 
 
−
=== Cloning a Repository ===
 
−
 
 
−
[[File:SourceTree-Clone.png|400px|thumb|right|Screenshot of cloning a repository in SourceTree]]
 
−
 
 
−
We will start by ''cloning'' a remote Git repository.
 
−
 
 
−
"Cloning" in Git is much like "checking out" in SVN: a remote copy of the repository is copied down to a local working directory.  '''Word of Caution:''' Git uses the term "checkout" to mean something different (which you will learn in Module 5), so ''do not'' use the words "clone" and "checkout" interchangeably.
 
−
 
 
−
If you wanted to clone a repository on the command line, the command would be:
 
−
 
 
−
<source lang="bash">$ git clone git://www.example.com/url-of-repo destination-path</source>
 
−
 
 
−
In SourceTree, go to '''File->New''', enter the remote repository URL and the destination path, and press "Clone".  The destination path is where you want the repository's files to be copied on your personal computer; the path should point to either an empty or a nonexistent directory on your personal computer.  In the screenshot, I am cloning the repository for the Fedora Linux kernel.
 
−
 
 
−
The remote repository will now be cloned onto your personal computer and all ready to use.
 
−
 
 
−
=== Committing ===
 
−
 
 
−
[[File:SourceTree-commit.jpg|400px|thumb|right|Screenshot of committing to a repository in SourceTree]]
 
−
 
 
−
When you make changes to files or add new files, you need to ''commit'' them to your working repository.  '''Note:''' When you "commit" something in SVN, it means that you are sending your changes to the remote repository.  This is ''not'' the case in Git, as we will see shortly.
 
−
 
 
−
To prepare your commit, press the "Commit" button in the SourceTree toolbar.  See the screenshot for an example.
 
−
 
 
−
Select the files you want to commit, and enter a commit message summarizing the changes you've made since your last commit.  When you're ready, press "Commit".
 
−
 
 
−
The command line equivalent of this would be:
 
−
 
 
−
<source lang="bash">
 
−
$ git add --all # adds untracked files to the repository and stages them for the commit
 
−
$ git commit -am "This is an example commit"
 
−
</source>
 
−
 
 
−
What happens now is that your changes are recorded ''in your own local working copy of the Git repository''.  The changes have '''''not''''' been saved in the remote repository.  You need to do this by ''pushing'' them.
 
−
 
 
−
=== Pushing ===
 
−
 
 
−
Git is different from SVN in that you can use it completely independently from a remote repository.  Thus, in Git, you need to perform the "commit" and "push" steps separately.
 
−
 
 
−
''After'' you perform a commit, you need to ''push'' your commit to your remote repository.  To do this, press the "Push" button in SourceTree, and then press "OK".  The command line equivalent is:
 
−
 
 
−
<source lang="bash">$ git push master # assuming your branch is still named the default "master"</source>
 
−
 
 
−
'''Note:''' You do not need to push after every commit.  You could have 4 or 5 commits before you actually perform the push.  When you push, all commits will be pushed.
 
−
 
 
−
=== Pulling ===
 
−
 
 
−
In SVN, if you want to download changes from your remote repository, you performed an ''update''.  In Git, you perform a ''pull''.
 
−
 
 
−
Pulling is (usually) as simple as pressing the "Pull" button in SourceTree.  The command line equivalent is:
 
−
 
 
−
<source lang="bash">$ git pull master</source>
 
−
 
 
−
'''Note:''' If you want to download the changes from the remote repository but not overwrite your current working copy, you can perform a ''fetch'' instead.
 
−
 
 
−
== Komodo Edit ==
 
−
 
 
−
[[File:Komodo.png|200px]]
 
−
 
 
−
This semester in CSE 330, we will be using Komodo Edit as our editor of choice.  We chose this editor because:
 
−
 
 
−
# It is free for everyone.
 
−
# It runs on Windows, Mac OS X, and Linux.
 
−
# It supports all of the languages we will be using in this class.
 
−
# It has a decent amount tools like auto-completion that will speed up your workflow.
 
−
 
 
−
=== Installation ===
 
−
 
 
−
Komodo Edit is already installed on the CEC lab machines.  If you prefer to use your personal computer, download it from the web site: http://www.activestate.com/komodo-edit
 
−
 
 
−
=== Connecting Komodo to your Remote Instance ===
 
−
 
 
−
With a little time spend on configurations and installing a plugin, Komodo Edit enables you to edit files on your desktop and then upload those files directly to your remote instance.
 
−
 
 
−
==== Adding SSH Public Key to Keychain ====
 
−
 
 
−
In order for Komodo to connect to your instance using SSH Public Key Authentication, you need to add your public key permanently to your SSH agent utility.  Instructions are below on how to do this in OS X and Windows.  ''You may skip to the next section if you are using password-based authentication.''
 
−
 
 
−
===== Mac OS X =====
 
−
 
 
−
# Open Terminal.
 
−
# Save your ''*.pem'' file to'' ~/.ssh/cse330.pem'':
 
−
#: <source lang="bash">$ mv /path/to/your/cse330.pem ~/.ssh/cse330.pem</source>
 
−
# Run the following command to add the key to your SSH agent:
 
−
#: <source lang="bash">$ ssh-add ~/.ssh/cse330.pem</source>
 
−
#: For more information on ''ssh-add'', see [http://www.openbsd.org/cgi-bin/man.cgi?query=ssh-agent the manual].
 
−
 
 
−
===== Windows =====
 
−
 
 
−
'' under construction ''
 
−
 
 
−
==== Configure your Server Settings in Komodo ====
 
−
 
 
−
Once you have your public key in your SSH agent (if applicable), follow these steps to tell Komodo how to connect to your remote instance.
 
−
 
 
−
# Open the Komodo Edit Preferences.  In Windows, this is '''Edit -> Preferences'''; in Mac OS X, this is '''Komodo -> Preferences'''.
 
−
# In the menu on the left, choose '''Servers'''.
 
−
# For Server Type, select '''SFTP'''.  (Since you have an SSH server running, you will be able to connect to your instance via SFTP.)
 
−
# Name it something like '''CSE 330 Cloud Instance'''.
 
−
# Enter the details of your instance, including your hostname and username.
 
−
#* If you are using Public Key Authentication, leave your password empty.
 
−
#* Unless you explicitly changed your SSH port, your port will be '''22'''.
 
−
# Your default path will be where you want to save the files you upload.  Generally, this should be the root of your web server.
 
−
#: [[File:Workflow-3.png]]
 
−
# Press '''Add''' to save the server information, and then press '''OK'''.
 
−
# '''IF YOU ARE USING PUBLIC KEY AUTHENTICATION''', you need to perform these additional steps:
 
−
## In the Komodo Edit Preferences, choose '''Environment'''.
 
−
## Find the information about your SSH environment:
 
−
##* '''Mac OS X:'''
 
−
##** Enter the command ''ssh-agent'' in Terminal.  You should see output similar to:
 
−
##**: <code>SSH_AUTH_SOCK=/tmp/ssh-xxxxx/agent.xxxxx; export SSH_AUTH_SOCK;
 
−
##**: SSH_AGENT_PID=xxxxx; export SSH_AGENT_PID;
 
−
##**: echo Agent pid xxxxx;</code>
 
−
##* '''Windows:'''
 
−
##** '' under construction ''
 
−
## Add the following environment variables in Komodo:
 
−
##* '''Name:''' SSH_AUTH_SOCK
 
−
##* '''Name:''' SSH_AGENT_PID
 
−
##: The values for these environment variables should be what you learned earlier in step 2.
 
−
## Press '''OK''' to save your changes.
 
−
# To test whether Komodo is able to successfully connect to your server, choose '''File -> Open -> Remote File'''.  Select ''CSE 330 Cloud Instance'' from the drop-down menu.  You should be able to see a list of files in your public_html directory.
 
−
 
 
−
==== Installing Upload Extension ====
 
−
 
 
−
Strangely, Komodo Edit does not come built-in with a mechanism to upload files from your computer to the server.  Fortunately, the community has released an add-on that gives us this feature.
 
−
 
 
−
# In Komodo, go to '''Tools -> Addons'''.
 
−
# Search for the add-on named ''Uploader''.
 
−
# Download and install it, and restart Komodo.
 
−
# In your Server preferences, create a new server (or rename an existing one) to be identical to your project name, but starting with a star.  For example, in my project named ''FileShare'', I would name the server ''*FileShare''.  The ''default path'' will be the location to which Komodo will upload your files.  '''Note: The path should be absolute.'''
 
−
# You can now select a file and go to '''File -> Upload''' to upload it to your server.
 
−
 
 
−
'''Tip:''' To make things quicker, you can bind a keyboard shortcut like ''ctrl/cmd+D'' to the Upload feature by following these steps:
 
−
 
 
−
# In Komodo preferences, choose '''Editor -> Key Bindings'''.
 
−
# Find the "Upload" command under "General".  Press in the "New Key Sequence" box, and press ''ctrl/cmd+D''.  Proceed through the on-screen instructions; you can name your new scheme ''Custom''.
 
−
# Save the preferences and restart Komodo.  You will now be able to upload files by simply pressing ''ctrl/cmd+D''.
 
−
 
 
−
 
 
−
 
 
−
== Starting a Project ==
 
−
 
 
−
When you start a new project, whether individual or group, you should create a new directory and a new Komodo Project file.
 
−
 
 
−
# Create a new directory in your repository named after the project you will be starting (e.g., ''fileshare'').
 
−
# In Komodo, choose '''Project -> New Project''' (or press shift-ctrl-N).
 
−
# Navigate to the directory you just created, enter a name for the project (e.g., ''FileShare.komodoproject''), and press "Save".
 
−
# You should now see a window like this:
 
−
#: [[File:Workflow-1.png]]
 
−
# Press ctrl-N (or cmd-N) to create a new file.  Save it in your project.  For example, you could create a page named ''index.html'' with the [[HTML and CSS#Quick and Easy Page Layout|Quick and Easy Page Layout]]:
 
−
#: [[File:Workflow-2.png]]
 
−
# Continue creating and editing the files from your site.
 
−
#: '''Tip:''' To change between files without using a mouse, press ctrl-PageUp and ctrl-PageDown (cmd instead of ctrl on OS X).  Many computers without explicit PageUp and PageDown buttons bind them to fn-Up and fn-Down.  For example, on a MacBook Pro, to change to the next tab, you would press three keys: fn+cmd+down.
 
−
# When you are ready, upload your files to your remote server.  To do this, go to '''File -> Save as Other -> Remote File'''.
 
−
 
 
−
[[Category:Module 2]]
 

Latest revision as of 18:25, 13 September 2024

The few minutes it takes to optimize your workflow will be the best few minutes you will spend in CSE 330 and your career. It is remarkable how many hours you can save just by using the best interface for text editing and deployment.

Overview

There are going to be several aspects to our workflow in CSE 330, and we will be switching between multiple different programs. Here is how they all fit together.

Running Commands

Mac OS X: You will be running commands on your local machine using Terminal, which you can find in your Applications -> Utilities folder. You will be connecting to your server via SSH.

Windows: You will be running commands on your local machine using WSL or Cygwin, which you will install in the next section. You will be connecting to your server via SSH.

Version Control

You will be using Git as your version control system. Git enables you to keep track of changes to your source code. If you're using VS Code, the Git integration makes is very user friendly.

Text Editing

Although we highly recommend VS Code, you may use Sublime Text, Komodo Edit, Notepad++ (for Windows), and TextWrangler (for Mac). You can also use an IDE, like PHPStorm, if you prefer.

File Transfer

In order to deploy your projects to your EC2 instance, you need some sort of method to transfer files from your local machine to your instance using SFTP. One option is using FileZilla, an FTP client, to transport files from your local computer to your server.

Flowchart

If you are more visual, this flowchart might help:

CSE-330-Workflow.png

The flowchart illustrates 3 methods to copy files to your cloud instance: via your text editor, via an external SFTP client, and by cloning your remote Git repository. We will be primarily using Option 2 in CSE 330 this semester.

In the flowchart, your Git Client is SourceTree and your Text Editor is Komodo. The SFTP Client could be FileZilla.

Installing a Terminal Emulator

Windows is based on a fundamentally different architecture than Linux, meaning that, if you want to run Linux programs on Windows, you'll have to enable or install a special terminal emulator for it. Mac users do not need to install a terminal emulator since MacOs itself is a derivative of Unix, which is an ancestor of Linux. The Terminal utility is all you need.

Windows Subsystem for Linux

Starting in Windows 10, Windows has the ability to emulate Linux, meaning that you can run Linux commands on Windows, without having to install Cygwin. To enable WSL, follow these instructions. Once those instructions are complete, you can open a shell by clicking on the Start menu, typing "Bash", then clicking on the entry labeled "Bash on Ubuntu on Windows".

Cygwin

Cygwin is a Linux Terminal emulator for Windows.

Installation

The CEC lab machines already have Cygwin installed. Follow these instructions to get Cygwin up and running on your personal PC.

  1. Run setup.exe from http://www.cygwin.com/
  2. Keep the Cygwin directory as the default, C:\cygwin. Choose a place like C:\cygwin\downloads to download the installation files. Choose any download site, although probably one with a .com suffix is best when you're in the US.
  3. In the "Select Packages" screen, select the following packages:
    • Net -> openssh
    • Net -> openssl
    • Python -> python3 (you will need this for Module 4)
    • Devel -> git (version control system)
    • One or more of the following:
      • Editors -> vim
      • Editors -> emacs
      • Editors -> nano
  4. Allow the installation to complete.

Throughout the rest of this course, when we say to "open a terminal", if you are on Windows, we mean to "open WSL/Cygwin".

Github

Github is a service that enables you to store your code for free in the cloud.

Creating a Github Account

Go to Github.org to create an account.

Important: Use your @wustl.edu e-mail address to create your account. This will put your account on the "educational plan", which removes some limitations of the free account.


Adding Your Key to Github

Not uploading your key is perfectly fine, and in many cases easier. Here are the instructions though, if you'd like

  1. First make sure that you've finished the SSH guide configuration instructions.
    We will use the same key for connecting to your remote server as for connecting to Github. (This is completely valid practice.)
  2. Go to the "SSH keys" section of the settings panel (Manage Account -> SSH Keys)
  3. Add a new key.
    • I recommend naming it after the computer you're using; if you're in the CEC, for example, you could name it "Warehouse Cygwin".
    • The contents of your key is the contents of your id_rsa.pub. It looks like ssh-rsa AAAAB3blahblahblahblah yourname@somedomain. It is exactly the same thing that you had to paste into the authorized_keys file on your EC2 instance.

You will now be able to push code to your Github repository using your personal SSH key.

FileZilla

You can use SFTP from the command line, or you can use any GUI file transfer client. All FTP clients I have seen also support SFTP. One popular FTP client is Filezilla.

  1. Download and install FileZilla from http://filezilla-project.org/download.php?type=client
    If you are using a lab computer, you can save it to your H drive.
  2. When you launch FileZilla, go to Edit→Settings or FileZilla→Preferences, and go to the SFTP options (under Connection). Click "Add keyfile…".
    • Choose your id_rsa file. OS X Tip: Press Shift+Command+Period to reveal hidden files in the file chooser window.
    • FileZilla will want to make a *.ppk file. A good place to save it would be the .ssh directory as id_rsa.ppk.
    When finished, press OK (not the red ×) to save your changes.
  3. Back on the main FileZilla screen, there are four fields: Host, Username, Password, and Port. Fill them in as follows:
    • Hostname: sftp://<ec2-xxx-xx-xx-xxx.compute-1.amazonaws.com/>
    • Username: The username you created on your server.
    • Password: Since you are using a key, you may leave this blank.
      If you changed your SSH settings to allow password-based authentication, then you could put your password in here.
    • Port: 22
  4. Then click Quickconnect. If everything is configured correctly, FileZilla should log into your server.

FileZilla.png

On the left and right of the FileZilla window, you can drag files between your computer and your server. Thus, you can edit a file in a text editor like Notepad++ on your decktop, and then upload it to your server by simply dragging it from the left pane to the right pane in FileZilla.

Retrieved from "http://classes.engineering.wustl.edu/cse330/index.php?title=Workflow&oldid=2929"