4
2016
Using Accord.NET with Unity
An article by 
While looking for Harris Corner Detection examples, without using openCV, founded this framework:
“The Accord.NET Framework is a machine learning framework combined with audio and image processing libraries completely written in C#”
Website: http://accord-framework.net
Github: https://github.com/accord-net/framework
*Note: There is updated tutorial here with images: https://github.com/accord-net/framework/wiki/Unity
How to get it running with Unity
– Open new empty project in Unity (I Used 5.5.0b10)
– Edit/Project Settings/Player, set Api compatibility Level to .NET2.0 (instead of .NET2.0 Subset)
– Download framework from https://github.com/accord-net/framework/releases ( I took v3.3.0 )
– Copy DLL’s from that package (from “Release/net35/” folder) into your projects Assets/Plugins/ folder
– Also copy System.Drawing.dll from your c: drive into “Assets/Plugins/” folder (should be version 2.0, I founded mine from “C:\Windows\Microsoft.NET\Framework\v2.0.50727\”)
– Create c# script and use the code below (*This was just a quick test to get it running, testing Gaussian filter on texture)
– See my scene setup screenshot below the source
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| using System.Collections; | |
| using UnityEngine; | |
| using Accord.Imaging.Filters; | |
| // More info: http://unitycoder.com/blog/2016/11/04/using-accord-net-with-unity/ | |
| public class AccordFrameworkTest : MonoBehaviour | |
| { | |
| public Renderer sourceRenderer; // our source texture is assigned on this gameobject | |
| public Renderer targetRenderer; | |
| void Start() | |
| { | |
| // get maintexture from source renderer | |
| var sourceTexture = (Texture2D)sourceRenderer.material.mainTexture; | |
| // create system bitmap | |
| System.Drawing.Bitmap accordImage = new System.Drawing.Bitmap(sourceTexture.width, sourceTexture.height); | |
| // copy our texture pixels to that system bitmap | |
| for (int x = 0; x < accordImage.Width; x++) | |
| { | |
| for (int y = 0; y < accordImage.Height; y++) | |
| { | |
| var c = (Color32)sourceTexture.GetPixel(x, y); | |
| var nc = System.Drawing.Color.FromArgb(c.r, c.g, c.b, c.a); | |
| accordImage.SetPixel(x, y, nc); | |
| } | |
| } | |
| // These 2 lines are the only Accord.NET code used here, create filter and then apply it to bitmap | |
| IFilter gaussianFilter = new GaussianBlur(2.0, 20); | |
| var resultsAccordImage = gaussianFilter.Apply(accordImage); | |
| // copy result pixels into our color32 array from system.drawing.bitmap | |
| var colors = new Color32[resultsAccordImage.Width * resultsAccordImage.Height]; | |
| for (int x = 0; x < resultsAccordImage.Width; x++) | |
| { | |
| for (int y = 0; y < resultsAccordImage.Height; y++) | |
| { | |
| var c = resultsAccordImage.GetPixel(x, y); | |
| colors[y * resultsAccordImage.Width + x] = new Color32(c.A,c.R, c.G, c.B); // colors are flipped in System.Drawing.Color | |
| } | |
| } | |
| // create new results texture from that color32 array | |
| var targetTexture = new Texture2D(sourceTexture.width, sourceTexture.height, sourceTexture.format, false); | |
| targetTexture.SetPixels32(colors); | |
| targetTexture.Apply(false); | |
| // assign it to another object | |
| targetRenderer.material.mainTexture = targetTexture; | |
| } | |
| } |
Image#1: Scene setup screenshot
– Add 2 Quads with different materials (use Unlit/Texture shader)
– Assign your source texture into QuadSource *Note: Texture importer settings must have [x] Read/Write enabled
– Assign those 2 quads into the script (which you have in some gameobject in scene) and hit Play!
*Main image: Using Gaussian filter from Accord.NET
Related Posts
12 Comments + Add Comment
Leave a comment
Recent posts
- [GreaseMonkey] Unity Forum Fixer
- UnityHub: Make Hub application background Translucent
- Customize SpriteShapeRenderer quality (but has issues)
- Editor tool: Copy selected gameobject’s names into clipboard as rows (for Excel)
- Editor tool: Replace string in selected gameobject’s names
- UnityHub: Enable built-in Login Dialog (no more browser login/logout issues!)
- Use TikTok-TTS in Unity (with WebRequest)
- Create Scene Thumbnail Image using OnSceneSaved & OnPreviewGUI
- webgl+javascript TTS
- Using Moonsharp (LUA) + Unity Webgl
- Using 3D gameobject prefabs with Unity Tilemap + NavMesh Surface
- Custom Unity Hub Project Template Preview Image/Video (using HTML+CSS in package description)
Connect
Recent Comments
- on Vector3 maths for dummies!
- on UnityHub: Make Hub application background Translucent
- on UnityHub: Make Hub application background Translucent
- on Install Android SDK+JDK+NDK for Unity (without AndroidStudio or Unity Hub)
- on Install Android SDK+JDK+NDK for Unity (without AndroidStudio or Unity Hub)
- on [Asset Store] Point Cloud Viewer & Tools
- on [Asset Store] Point Cloud Viewer & Tools
- on ffmpeg stream raw video into Unity Texture2D
Thanks for bringing this to my attention. I don’t need it for Unity but we do occasionally use OpenCV with at my job and it can be a struggle using it with .NET (even with the popular wrappers available). This looks like an interesting potential alternative.
[…] 少し調べたら、Unity5.5でAccord.NETを動作させるチュートリアルがあった。↓ Using Accord.NET with UnityWhile looking for Harris Corner Detection examples, without using openCV, founded this framework: […]
Getting Started With OpenCvSharp 3
https://www.codeproject.com/tips/1085960/getting-started-with-opencvsharp
“Anyone know a machine learning library for C sharp?”
https://forum.unity3d.com/threads/anyone-know-a-machine-learning-library-for-c-sharp.470614/
http://numl.net/
http://www.heatonresearch.com/encog/
Hello, When I followed the above steps install the Accord.net, but it has one problem like the following:
TypeLoadException: Could not load type ‘Accord.DirectSound.AudioDeviceInfo’ from assembly ‘Accord.Audio.DirectSound, Version=3.3.0.0, Culture=neutral.
So May I ask how to solve this problem? Thanks
seems to happen in 2017.x, but not in that 5.5.x.. would be interesting to know what changed.
But I tried 5.5.0f3 at first, it still has similar problem. I used accord. Net at version 3.3, and follow your above steps. So would you please help me what I should fix? Thank you so much!
in 5.5.3p4 it gives notification message (not error like in 2017) “SpritePacker failed to get types from Accord.Audio.DirectSound, Version=3.3.0.0, Culture=neutral, PublicKeyToken=fa1a88e29555ccf7. Error: The classes in the module cannot be loaded.
UnityEditor.Sprites.Packer:GetSelectedPolicyId()”
but only if spritepacker is enabled.. could always delete that directsound dll for now
The SpritePacker error is an unhelpful error that indicates that Assembly.GetTypes() failed, which means that some dependency of Accord.Audio.DirectSound is missing.
What dependency? The error doesn’t say. But I resolved a similar issue when I tried to build the project for iOS, at which point I got an error that told me exactly what was missing.
I got this working on macOS after 10+ hours of trying. Here’s my documentation of what I did: https://github.com/accord-net/framework/issues/799#issuecomment-324157507.
Curiously, unless some of the items I mention in the bottom bullets were necessary, it turned out to be the steps from this post, but without needing System.Drawing.dll. I originally got thrown off because I couldn’t find System.Drawing.dll on my computer, but that link above also points to where I eventually found it (though I didn’t keep it in Assets).
there is some updated tutorial here,
Installing the Accord.NET Framework in Unity
https://github.com/accord-net/framework/wiki/Unity
U really expect me to code to run Accord.Net ? Madman’s developers developing Libraries to be used only by developers. No install packages. Why don’t u say so loud and clear. Why i have to install oversize’d software packages in order to sun few lines of code mainly CLI driven with commands that needs to be learn first ? Why i have to code it in order to be able to run it ? If this approach would be used from Microsoft then no user will use it. That’s why such packages and Libraries are widely learned and used !? U cant even start to learn about ANN’s what so ever if u cant run it ? Right ?