GIT Distributed Integration Workflows

Background

• team members in different geographical locations; no face time.
• different timezone; work independently.
• one US-based integration manager who is customers, analysts, and QA facing
• integration manager has read/write access to official repository
• team members have read access to official repository
• each team member manages his own private and public repositories.
• company/project.git is a shared bare repo

Workflows

• Team member will
• set up a personal public repository accessible to integration manager
• su - member1
• cd /home/member1
• mkdir project.git
• cd project.git
• git --bare init --shared

• set up a private repository by cloning of the official repository
• cd ~/repos/git
• git clone https://github.com/company/project.git
• cd ~/repos/git/project
• git config --global user.name "Your Name"
• git remote add personal_public member1@<server name>:project.git

• create a new local branch in preparation of making changes
• git checkout -b dev

• commit changes to local branch
• git add .
• git commit -m "comment"

• push local branch to personal public repository
• git push personal_public dev:dev

• tag release
• git tag v1.2
• git push personal_public v1.2
• email integration manager of integration request

• Integration Manager
• set up a personal private repository by cloning of the official repository
• cd ~/repos/git
• git clone https://github.com/company/project.git
• cd ~/repos/git/project
• git remote add member1 member1@<server name>:/home/member1/project.git

• fetch and merge remote branch
• git fetch member1
• git branch -a
• git log -p master..member1/dev
• git merge --log member1/dev
• fetch and merge individual files
• git fetch member1
• git branch -a
• git log -p master..member1/dev
• git checkout member1/dev <file name>
• fetch, exclude individual files from merge
• git fetch member1
• git checkout dev
• dev is a remote branch on member1
• cannot "git checkout member1/dev" as that results in detached HEAD
• git checkout HEAD~1 <file name>
• git commit -m "exclude files from merge"
• git checkout master
• git merge --log dev

• push to the official github repository
• git push
• push tag
• git tag -l
• git push origin v1.2
• push to heroku
• ssh-add -D
• ssh-add ~/.ssh/id_rsa
• cd ~/deploy/git
• git clone git@heroku.com:project.git -o heroku
• cd  ~/deploy/git/project
• git remote add github https://github.com/company/project.git
• git fetch github
• git log -p master..github/master
• git merge --log github/master
• git push heroku

• manual packaging and release management
• set up a remote release repository
• mkdir official.git
• cd official.git
• git --bare init --shared
• package a local dev branch

• git checkout dev
• git branch v1.0
• git checkout v1.0
• bundle package --all
• git add vendor/cache
• git commit -m "v1.0 release"
• push official release
• git remote add official-releases integrator@release-server:official.git
• git push official-releases v1.0:v1.0
• clean slate deployment of an official release
• mkdir new-release
• cd new-release
• git clone integrator@release-server:official.git .
• git checkout v1.0
• bundle install --local

IP

Organization: WSI

Program: IP

Form 1 Question 1

Question Name: Do you have a policy that prohibits unauthorized access to the system?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 2

Question Name: Do you have new employees execute agreement not to disclose or use trade secrets from former employees?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 3

Question Name: Do you properly secure trade secrets to prevent forfeiture of rights?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 4

Question Name: Do you change all default vendor passwords?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 5

Question Name: Do you have evidence of trademark use?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 6

Question Name: Do you have evidence of firewall security audit?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 7

Question Name: Do you have a password policy?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 8

Question Name: Do you have an access control policy?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 9

Question Name: Do you monitor trademark infringement and unfair competition?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 10

Question Name: Do you keep record of employee training?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 11

Question Name: Do you have incident handling policy?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 12

Question Name: Do you have designated incident response team?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

Form 1 Question 13

Question Name: Do you and/or your employees fail to report any security-related incidents?

Answer Type: Select Single 'Yes', Select Single 'No', Select Single 'Other'

rake cron job

automatically run every minute

vi /etc/crontab

* * * * * root cd /var/www/data.tangentenergy.com/current && /usr/local/bin/rake RAILS_ENV=production poll_pjm_unack > /tmp/pjm_unack

set up rake task

vi /rails-app/current/lib/tasks/my.rake

desc "scrap external data"
task :poll_events => :environment do
  ExternalData.poll_events
end

RVM

Installation

• download RVM installer
• edit bash_profile

[[ -s  "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm"  # Load RVM into a shell session *as a function*

• reload shell
• type rvm | head -1
• should show rvm is a function

Installation on mac mini 10.8

• to download&install RVM installer to /Users/aleung/.rvm/ on mac mini
• curl -L https://get.rvm.io | bash

• to set up PATH in shell
• source /Users/aleung00c/.rvm/scripts/rvm

More Commands

• rvm -v
• rvm list
• rvm list known
• rvm info
• rvm gemdir
• switch to gem directory for current ruby
• rvm gemdir system
• switch to system gem directory
• rvm gemdir system user
• rvm use system
• to undo rvm environment changes

Ruby 2.0.0 / Rails 4.0.0 Installation

• rvm get stable --auto-dotfiles
• rvm install 2.0.0
• rvm use 2.0.0
• rvm gemset create rails4.0.0
• rvm gemset use rails4.0.0
• gem install rails -v 4.0.0

 Ruby 1.9.3 Installation

• rvm install ruby-1.9.3-p327

Ruby 1.9.2 / Rails 3.0.9 Installation

• rvm install 1.9.2
• rvm --default use 1.9.2
• ruby -v
• which ruby
• gem install rails -v 3.0.9

Ruby 1.8.7 / Rails 3.0.0 Installation

• rvm install 1.8.7
• rvm gemset create newproject
• rvm 1.8.7@newproject
• rvm @global do gem uninstall bundler
• gem install bundler -v '~>1.0.0'
• we needed to remove bundler 1.1.3 in global gemset, in order to install bundler 1.0.0 specified in project Gemfile

Ruby 1.8.6 / Rails 2.1.1 Installation

• rvm install 1.8.6
• rvm gemset create legacy-gemset
• rvm 1.8.6@legacy-gemset
• gem install rails --version 2.1.1
• gem install sqlite3-ruby --version 1.2.4
• gem install ruby-net-ldap --version 0.0.4

Additional Gemset Creation

• rvm gemset create experiment-gemset
• rvm 1.9.2@experiment-gemset
• gem install fastercsv
• rvm gemset copy 1.9.2 1.9.2@experiment-gemset
• rvm gemset use experiment-gemset

Rails 3.0.9 App Creation

• rvm use 1.9.2
• rails -v
• to verify and confirm we are using rails version 3.0.9 as default gemset
• cd ~/ruby1.9.2/rails_projects/
• rails new new-rails-app

Rails 2.1.1 App Creation

• rvm use 1.8.6
• rails -v
• to verify and confirm that we are using rails version 2.1.1 as default gemset
• cd ~/ruby1.8.6/rails_projects/
• rails new-rails-app
• cd new-rails-app
• rails s

RSpec (Ruby 1.9.2 / Rails 3.0.9)

• gem install rspec-rails
• cd ~/ruby1.9.2/rails_projects/new-rails-app
• add to Gemfile

group :test, :development do
  gem "rspec-rails", "~> 2.0"
end

• rails generate rspec:install
• rails generate model PricingPlan sku:string pricing:string
• rake db:migrate RAILS_ENV=test
• rspec spec/models/pricing_plan_spec.rb

Experiment with new Savon

• rvm gemset create new_savon
• rvm gemset copy 1.9.2@production 1.9.2@new_savon
• rvm gemset use 1.9.2@new_savon
• gem list savon

*** LOCAL GEMS ***

savon (0.9.6)
savon_spec (0.1.6)

• view Gemfile.lock

    savon (0.9.6)
      akami (~> 1.0)
      builder (>= 2.1.2)
      gyoku (>= 0.4.0)
      httpi (~> 0.9)
      nokogiri (>= 1.4.0)
      nori (~> 1.0)
      wasabi (~> 2.0)

• bundle update savon 
• gem list savon

*** LOCAL GEMS ***

savon (1.1.0, 0.9.6)
savon_spec (0.1.6)

• view Gemfile.lock

    savon (1.1.0)
      akami (~> 1.2.0)
      builder (>= 2.1.2)
      gyoku (~> 0.4.5)
      httpi (~> 1.1.0)
      nokogiri (>= 1.4.0)
      nori (~> 1.1.0)
      wasabi (~> 2.5.0)

Rails 3.2

• rvm gemset create development_32
• rvm gemset use development_32
• bundle install

Rolify

• rvm gemset create rolify
• rvm gemset use rolify
• add to Gemfile

  gem 'rolify'

• bundle install
•  

pgp on mac

Install GPG Suite from https://gpgtools.org/

To check the detached signature (a.asc) against our release (a.tar.gz).
Option 1: Without importing KEYS file

$ gpg --verify pgp.asc commons-daemon-1.0.15-bin.zip
gpg: Signature made Thu Mar 28 08:54:06 2013 EDT using DSA key ID 0D498E23
gpg: requesting key 0D498E23 from hkps server hkps.pool.sks-keyservers.net
gpg: key 0D498E23: public key "Mladen Turk (Default signing key) <mturk@apache.org>" imported
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 4 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 4u
gpg: next trustdb check due at 2021-04-28
gpg: Total number processed: 1
gpg: imported: 1
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: F7DA 48BB 64BC B84E CBA7 EE69 35CD 23C1 0D49 8E23

Option 2: To import KEYS file

Download KEYS file
$ gpg --import KEYS
$ gpg --list-keys
$ gpg --verify pgp.asc commons-daemon-1.0.15-bin.zip
gpg: Signature made Thu Mar 28 08:54:06 2013 EDT using DSA key ID 0D498E23
gpg: Good signature from "Mladen Turk (Default signing key) <mturk@apache.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: F7DA 48BB 64BC B84E CBA7 EE69 35CD 23C1 0D49 8E23

To trust a key

$ gpg --edit-key 0x0D498E23 trust

To checksum

$ md5 commons-daemon-1.0.15-bin.zip
MD5 (commons-daemon-1.0.15-bin.zip) = b3bc9ed9cf40e08868d36a0c1161d71e

rbenv on macmini

instruction

- https://github.com/rbenv/rbenv#installation

install git

- https://sourceforge.net/projects/git-osx-installer/files/

Check out rbenv into ~/.rbenv.

- $ git clone https://github.com/rbenv/rbenv.git ~/.rbenv

Optionally, try to compile dynamic bash extension to speed up rbenv. Don't worry if it fails; rbenv will still work normally:

- $ cd ~/.rbenv && src/configure && make -C src

Add ~/.rbenv/bin to your $PATH for access to the rbenv command-line utility.

$ echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bash_profile

Run ~/.rbenv/bin/rbenv init for shell-specific instructions on how to
initialize rbenv to enable shims and autocompletion.

# Load rbenv automatically by appending
# the following to ~/.bash_profile:
eval "$(rbenv init -)"

Restart shell and test

$ type rbenv
#=> "rbenv is a function"

Install ruby-build

git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build 

List all available versions

$ rbenv install -l

install a ruby version

$ rbenv install 2.4.1

set up project

$ cd ~/work
$ git clone https://github.com/r509/r509.git
$ cd r509
$ rbenv version
$ rbenv local

set ruby version

$ rbenv local 2.4.1
$ rbenv local

initial ruby version setup

$ gem install bundler
$ gem env home

initial project setup

$ bundle install
$ bundle exec gem list
$ gem list 

setup project

$ cd ~/work
$ git clone https://github.com/r509/r509-ca-http.git
$ cd r509-ca-http
$ rbenv local 2.4.1

initial project setup

$ bundle install

setup project

$ cd ~/work
$ git clone https://github.com/r509/r509-middleware-validity.git
$ cd r509-middleware-validity
$ rbenv local 2.4.1

initial project setup

$ bundle install

setup project

$ cd ~/work
$ git clone https://github.com/r509/r509-middleware-certwriter.git
$ cd r509-middleware-certwriter
$ rbenv local 2.4.1

initial project setup

$ bundle install

initial redis download and setup
- instructions http://jasdeep.ca/2012/05/installing-redis-on-mac-os-x/

$ cd ~/work
$ tar xfz redis-3.2.8.tar.gz
$ cd redis-3.2.8
$ sudo make test
$ sudo make
$ sudo mv src/redis-server /usr/local/bin
$ sudo mv src/redis-cli /usr/local/bin
$ mkdir ~/.redis
$ touch ~/.redis/redis.conf
$ redis-server

initial binary project setup

$ mkdir ~/work/r509-howto-ca
$ cd ~/work/r509-howto-ca
$ rbenv local 2.4.1
$ bundle init
- edit Gemfile to include the followings
gem 'r509'
gem 'r509-ca-http'
gem 'r509-middleware-validity'
gem 'r509-middleware-certwriter'
$ bundle install

Rails Development on Windows

install Git

• download Git-1.7.4-preview20110204.exe from http://code.google.com/p/msysgit/
• git config --global user.name "Your Name"
• git config --global user.email "Your Email"

install MONGO

• download mongodb-win32-i386-1.6.5
• unzip to c:\mongo
• mkdir c:\data
• mkdir c:\data\db
• to start server
• c:\mongo\bin\mongod.exe
• to start shell
• c:\mongo\bin\mongo.exe 
• db
• exit

install PostgreSQL

• download postgresql-8.3.13-1-windows.exe

install MySQL

• download mysql-noinstall-5.0.91-win32.zip
• 5.1 and above would not work with Ruby
• unzip to c:\mysql-5.0.91-win32

install Ruby

• download rubyinstaller-1.8.7-p302.exe from http://www.ruby-lang.org/en/downloads/
• note: Rails 3 only support 1.8.7 and 1.9.2
• Ruby 1.9.2 has dependency problem with mongrel gem, so use 1.8.7 for now
• for this installation guide, assume installed location = c:\Ruby1.8.7
• create c:\Ruby1.8.7\setenv.bat
• set path=%path%;C:\Ruby187\bin
• set path=%path%;C:\Program Files\Git\cmd
• set path=%path%;C:\mysql-5.0.91-win32\lib
• alternatively, modify the system path to include the above

inspect Gems

• gem update --system
• gem list --local
• gem sources

install Rack

• gem install rack

install Rails

• gem install rails --version 3.0.0
• rails -v

install MySQL driver

• bundle config build.mysql -- --with-mysql-include=c:\mysql-5.0.91-win32\include --with-mysql-lib=c:\mysql-5.0.91-win32\lib
• gem install mysql
• copy c:\mysql-5.0.91-win32\bin\libmysql.dll to c:\Ruby187\bin

install postgres driver

• gem install pg --version 0.9.0

install pagination add-on

• gem install will_paginate

install acts_as_versioned add-on

• gem install acts_as_versioned

clone existing project from server

• mkdir c:\Ruby1.8.7\rails_projects
• cd c:\Ruby187\rails_projects
• git init
• git clone user2@host:/pub/proj/project.git myclone
• bundle check
• if gems are packaged already in vendor/cache
• bundle install --local
• otherwise to install gems from rubygems.org
• bundle install
• after installing new gems or updating to newer versions
• bundle update

create new Rails application (in DOS prompt)

• create basic project template
• cd  c:\Ruby187\rails_projects
• rails new new-rails-app
• cd  c:\Ruby187\rails_projects\new-rails-app
• specify dependencies in a Gemfile in project's root
• c:\Ruby187\rails_projects\new-rails-app\Gemfile
• gem 'sinatra'
• gem 'mysql'
• gem 'activerecord'
• gem 'rest-client'
• gem 'json', '1.4.6'
• cannot use json 1.5.1 depends on ruby 1.9.1
• install bundle
• bundle config
• bundle install
• bundle show
• package gems to prepare for deployment
• bundle package
• create simple test
• rails generate scaffold User name:string email:string
• rails generate migration Timelines
• rails generate migration AddDetailsToTimelines timesheet_task_id:integer
• rake db:migrate
• initialize database
• rake db:create
• rake db:schema:load
• rake db:seed
• or rake db:setup to do the above 3
• start server
• rails server (or rails s)
• test in browser
• http://localhost:3000/users
• unit test
• cd test
• rake db:test:load
• ruby unit/user_test.rb
• check in
• git init
• specify files we do not want to track
• create c:\Ruby187\rails_projects\new-rails-app\.gitignore
• log/ 
• .bundle/
• git add .
• git commit -m "initial commit"

add new project to github

• cd c:\Ruby187\rails_projects\new-rails-app
• git remote add github git@github.com:mozgroup/2011.git  
• git push github master

install SQLITE3 DLL

• from http://sqlite.org/download.html, download sqlite-3_7_2.zip and sqlitedll-3_7_2.zip
• extract inside C:\Ruby1.8.7\bin

install git in RadRails

• right click in Plugins Manager to install git
• HOST: git server e.g. git.mycompany.com
• REPOSITORY PATH: /home/scone/project.git
• URI: git+ssh://scone@git.mycompany.com/home/scone/project.git
• PROTOCOL: git+ssh

in order to support different databases in different environments

• edit C\Ruby187\rails_project\new_rails_app\Gemfile
• group :development, :test do gem 'sqlite3-ruby', :require => 'sqlite3' end
• group :production, :staging do gem 'pg', '0.9.0' end
• note: 0.10.0 not working on windows
• install pg (postgresql driver gem)
• install postgres from
• bundle config
• bundle config build.pg --with-pg-include=c:\Program Files\PostgreSQL\8.3\include
• bundle config config.pg --with-pg-config=c:\Program Files\PostgreSQL\8.3\bin
• bundle

install heroku

• gem install heroku
• heroku keys:add
• assumimg exisitng public and private keys in user home folder
• heroku create new_rails_app
• heroku config:add BUNDLE_WITHOUT="development test"
• git push heroku master

set up heroku database schema

• heroku run rake db:migrate  --trace --app app_name

inspect heroku database setup

• heroku console
• ActiveRecord::Base.connection.tables 

populate heroku database from local development database

• gem install taps
• heroku db:push

inspect heroku

• more .git\config
• heroku list
• git remote show heroku
• gitk --all
• heroku logs