Matlab Fanuc Kinect
From ESE497 Wiki
Jump to navigationJump to searchSource Code for communication between Kinect Camera and Fanuc Robot
Special thanks for Andrew Schoer, Joshua Remba, Pradosh Kharel and Ian Sekiguchi for contributing to this page.
- Before you launch Matlab, as administrator:
- Open a Command Prompt
- cd c:\Program Files (x86)\Windows Resource Kits\Tools
- Type C:\Program Files (x86)\Windows Resource Kits\Tools>ntrights -u ACCOUNTS\<wustl key> +r SeLockMemoryPrivilege
- Use H:\* instead of \\warehouse.* for your Matlab working directory. If you browse to your home directory on the H: drive and double click on Kinect2SharedMem2Matlab.m this will happen automatically.
- Zip file
Source Code Description:
- Calibration.m:
- % Script to calibrate the Kinect Camera to the Fanuc Robot
- % Moves robot to 64 locations and locates X in Kinect image and then computes
- % the transformation matrix between the Kinect and Robot. Creates TransMatrix.mat
- % which contains transformation matrix T. Stores image data for post processing in
- % calibrationInput.mat
- Kinect2SharedMem2Matlab.m:
- % Example script to acquire images from the Kinect camera in Matlab
- % -------------------------------------------------------------------------
- % Kinect class uses a slightly modified Microsoft Kinect SDK application
- % DepthWithColor-D3 and SharedMemDLL.dll to stream images from the Kinect camera
- % to this script. This script initializes the shared memory and then launches
- % DepthWithColor-D3.exe. DepthWithColor-D3 connects to the **% shared memory
- % and starts streaming RGB and depth images to it.
- % This is preferable to using the KinectForWindowsExample.m because
- % DepthWithColor-D3 aligns the RGB and Depth images.
- Matlab2FanucExample.m: Send a series of movements to the Fanuc Robot using TCP/IP. Run LabVIEWMatlabFanuc\LabVIEWMatlabFanuc.vi and LV_COM_FRM on the Fanuc Robot first.
- % Example script to move robot with Matlab commands. Runs on computer
- % connected to Robot or on a remote computer
- % -------------------------------------------------------------------------
- % Launch the teaching pendant program LV_COM_FRM before launching this script.
- % See: http://classes.engineering.wustl.edu/ese497/index.php/Fanuc_How_Tos,_FAQ,_etc.#FAQ
- % for detailed instructions.
- % -------------------------------------------------------------------------
- % Messages are sent from this script to the LabVIEWMatlabFanuc.exe to
- % communicate the new position of the robot. The first 4 bytes of
- % the message are the length of the message in ASCII and that the message
- % is floating point numbers delimited by commas. You need to wait until it
- % receives an 'Ack' message back from LabVIEW.
- % -------------------------------------------------------------------------
- % If this is the computer that also communicates with the robot, then set
- % host to 'localhost'. In this case, the script will launch
- % LabVIEWMatlabFanuc.exe. This is the application that receives message
- % over TCP/IP from this script and communicates the coordinates to the
- % program running on the robot.
- % If this is NOT the computer that communicates with the robot, then set
- % host to 'fanucomp.seasad.wustl.edu' or whatever the IP is for that
- % computer. You will have to manually run LabVIEWMatlabFanuc.exe on that
- % computer and set the IP to this computer's address. You have to stop the
- % application to change the IP. The application also starts minimized so
- % you'll have to find it in the taskbar.
- % -------------------------------------------------------------------------
- % Matlab to LabVIEW communication based on
- % http://www.mathworks.com/matlabcentral/fileexchange/42567-matlab-and-labview-data-exchange-over-tcp-ip
- % Modified to pre-pend the length the the xmit string and to return the
- % received message as a string
- % -------------------------------------------------------------------------
- TestCalibration.m:
- % Script to test the calibration of the Kinect Camera to the Fanuc Robot
- % Moves robot to random locations and locates X in Kinect image.
- % The calculated position of the Fanuc, based on the Kinect and Transformation matrix,
- % is then compared to the position that the Fanuc was sent to.
- Miscellaneous Functions
- Kinect.m:
- % Class for communicating with the Kinect Camera. Uses SharedMemDLL.dll and DepthWithColor-D3D.
- proj2cart.m: Convert from Kinect projection to cartesian coordinates. Best FOVx,FOVy is 63.1,49.6.
- % Converts from Camera Pixel Coordinates and Real World Depth to
- % RealWorld x,y,z
- % In order to convert Kinect Coordinates to FANUC Coordinates:
- % 1.) Convert Kinect Coordinates to Cartesian using proj2cart function
- % 2.) Transponse Cartesian Coordinates to Column Vectors
- % 3.) Append One as 4th element in Column Vector
- % 4.) Multiply Column Vector(s) by Transform Matrix, T
- Kinect Projection to Cartesian Conversion.docx
- % Details of proj2cart.m function
- absor.m
- % Tool for calibrating 2 coordinate systems from
- % Author: Matt Jacobson
- % Copyright, Xoran Technologies, Inc. http://www.xorantech.com
- CreateTemplates.m
- % Creates templates for X and + images
- DepthFilter.m
- % Median filter for depth
- Double2CSV.m
- % Converts array of doubles to commma separated string
- exchangeData.m
- % transmits data over TCP/IP to LabVIEW program. Used to send robot coordintates.
- DepthWithColor-D3D
- % Slightly modified Microsft Kinect SDK example based on their example code DepthWithColor-D3D
- % This is preferable to the KinectForWindows.m example because it solves the RGB/Depth registration
- % issues.
- LabVIEW2Fanuc
- % LabVIEW source code and Application to pass commands from Matlab over TCP/IP to the Fanuc Robot using
- % the NI Ethernet/IP toolkiit. LV_COM_FRM must be launched on the Teaching Pendant to complete the communication
- SharedMem\SharedMemDLL
- % Microsoft C++ built DLL to pass images in a large shared memory between 2 applications
- TeachingPendantProgram
- % Source code for LV_COM_FRM that is launched from the teaching pendant to communicate with the LabVIEW2Fanuc Application
- RGB Depth Registration.docx
- % Document with pictures of depth registration using KinectForWindowsExample.m and my Kinect class
- Kinect.m:
