Matlab Fanuc Kinect
From ESE497 Wiki
Jump to navigationJump to searchSource Code for communication between Kinect Camera and Fanuc Robot
- Special thanks for Andrew Schoer, Joshua Remba, Pradosh Kharel and Ian Sekiguchi for contributing to this page.
- Before you launch Matlab, as administrator:
- Open a Command Prompt
- cd c:\Program Files (x86)\Windows Resource Kits\Tools
- Type C:\Program Files (x86)\Windows Resource Kits\Tools>ntrights -u ACCOUNTS\<wustl key> +r SeLockMemoryPrivilege
- Use H:\* instead of \\warehouse.* for your Matlab working directory. If you browse to your home directory on the H: drive and double click on Kinect2SharedMem2Matlab.m this will happen automatically.
- Zip file
Source Code Description:
- Calibration.m:
- % Script to calibrate the Kinect Camera to the Fanuc Robot
- % Moves robot to 64 locations and locates X in Kinect image and then computes
- % the transformation matrix between the Kinect and Robot. Creates TransMatrix.mat
- % which contains transformation matrix T. Stores image data for post processing in
- % calibrationInput.mat
- Kinect2SharedMem2Matlab.m:
- % Example script to acquire images from the Kinect camera in Matlab
- % -------------------------------------------------------------------------
- % Kinect class uses a slightly modified Microsoft Kinect SDK application
- % DepthWithColor-D3 and SharedMemDLL.dll to stream images from the Kinect camera
- % to this script. This script initializes the shared memory and then launches
- % DepthWithColor-D3.exe. DepthWithColor-D3 connects to the **% shared memory
- % and starts streaming RGB and depth images to it.
- % This is preferable to using the KinectForWindowsExample.m because
- % DepthWithColor-D3 aligns the RGB and Depth images.
- Matlab2FanucExample.m: Send a series of movements to the Fanuc Robot using TCP/IP. Run LabVIEWMatlabFanuc\LabVIEWMatlabFanuc.vi and LV_COM_FRM on the Fanuc Robot first.
- % Example script to move robot with Matlab commands. Runs on computer
- % connected to Robot or on a remote computer
- % -------------------------------------------------------------------------
- % Launch the teaching pendant program LV_COM_FRM before launching this script.
- % See: http://classes.engineering.wustl.edu/ese497/index.php/Fanuc_How_Tos,_FAQ,_etc.#FAQ
- % for detailed instructions.
- % -------------------------------------------------------------------------
- % Messages are sent from this script to the LabVIEWMatlabFanuc.exe to
- % communicate the new position of the robot. The first 4 bytes of
- % the message are the length of the message in ASCII and that the message
- % is floating point numbers delimited by commas. You need to wait until it
- % receives an 'Ack' message back from LabVIEW.
- % -------------------------------------------------------------------------
- % If this is the computer that also communicates with the robot, then set
- % host to 'localhost'. In this case, the script will launch
- % LabVIEWMatlabFanuc.exe. This is the application that receives message
- % over TCP/IP from this script and communicates the coordinates to the
- % program running on the robot.
- % If this is NOT the computer that communicates with the robot, then set
- % host to 'fanucomp.seasad.wustl.edu' or whatever the IP is for that
- % computer. You will have to manually run LabVIEWMatlabFanuc.exe on that
- % computer and set the IP to this computer's address. You have to stop the
- % application to change the IP. The application also starts minimized so
- % you'll have to find it in the taskbar.
- % -------------------------------------------------------------------------
- % Matlab to LabVIEW communication based on
- % http://www.mathworks.com/matlabcentral/fileexchange/42567-matlab-and-labview-data-exchange-over-tcp-ip
- % Modified to pre-pend the length the the xmit string and to return the
- % received message as a string
- % -------------------------------------------------------------------------
- TestCalibration.m:
- % Script to test the calibration of the Kinect Camera to the Fanuc Robot
- % Moves robot to random locations and locates X in Kinect image.
- % The calculated position of the Fanuc, based on the Kinect and Transformation matrix,
- % is then compared to the position that the Fanuc was sent to.
- Miscellaneous Functions
- Kinect.m:
- % Class for communicating with the Kinect Camera. Uses SharedMemDLL.dll and DepthWithColor-D3D.
- proj2cart.m: Convert from Kinect projection to cartesian coordinates. Best FOVx,FOVy is 63.1,49.6.
- % Converts from Camera Pixel Coordinates and Real World Depth to
- % RealWorld x,y,z
- % In order to convert Kinect Coordinates to FANUC Coordinates:
- % 1.) Convert Kinect Coordinates to Cartesian using proj2cart function
- % 2.) Transponse Cartesian Coordinates to Column Vectors
- % 3.) Append One as 4th element in Column Vector
- % 4.) Multiply Column Vector(s) by Transform Matrix, T
- Kinect Projection to Cartesian Conversion.docx
- % Details of proj2cart.m function
- absor.m
- % Tool for calibrating 2 coordinate systems from
- % Author: Matt Jacobson
- % Copyright, Xoran Technologies, Inc. http://www.xorantech.com
- CreateTemplates.m
- % Creates templates for X and + images
- DepthFilter.m
- % Median filter for depth
- Double2CSV.m
- % Converts array of doubles to commma separated string
- exchangeData.m
- % transmits data over TCP/IP to LabVIEW program. Used to send robot coordintates.
- DepthWithColor-D3D
- % Slightly modified Microsft Kinect SDK example based on their example code DepthWithColor-D3D
- % This is preferable to the KinectForWindows.m example because it solves the RGB/Depth registration
- % issues.
- LabVIEW2Fanuc
- % LabVIEW source code and Application to pass commands from Matlab over TCP/IP to the Fanuc Robot using
- % the NI Ethernet/IP toolkiit. LV_COM_FRM must be launched on the Teaching Pendant to complete the communication
- SharedMem\SharedMemDLL
- % Microsoft C++ built DLL to pass images in a large shared memory between 2 applications
- TeachingPendantProgram
- % Source code for LV_COM_FRM that is launched from the teaching pendant to communicate with the LabVIEW2Fanuc Application
- RGB Depth Registration.docx
- % Document with pictures of depth registration using KinectForWindowsExample.m and my Kinect class
- Kinect.m:
