Desktop capture streamed into Unity material texture2D<\/figcaption><\/figure>\n\n\n\nNext steps<\/strong>
– TCP version maybe fixes missed frames\/out of sync problems
– Use proper container mpegts or so, to fix issues with udp missed frames etc. but difficult to decode mpeg4 payload data(?)<\/p>\n\n\n\n$$$ Reward<\/strong> $$$
– First one to post working pull-request (or separate repo) for mpegts\/mp4\/mpeg4 or handle other proper video stream parsing inside unity (without plugins, or only open source plugins) wins 50usd\/paypal<\/s>, 100usd\/paypal<\/strong> : D because i think this could be useful open source tool for the community (to be able to easily stream data from ffmpeg)
– Requirements, parse this data into texture2d:
ffmpeg -f gdigrab -i desktop -r 1 -f mpegts udp:\/\/127.0.0.1:8888
\/\/ or more options
ffmpeg -f gdigrab -i desktop -pixel_format rgb24 -video_size 512×512 -vf scale=512:512 -framerate 1 -r 1 -f mpegts dp:\/\/127.0.0.1:8888
\/\/ it doesnt have to be plain UDP, can use rtp, tcp or others too, can use different pixel format too<\/p>\n","protected":false},"excerpt":{"rendered":"Small proof-of-concept idea about streaming raw video from commandline ffmpeg into Unity, using UDP streaming. It works ok for small resolution and low fps rate, but breaks\/goes out of sync if any udp data goes missing. Sources:https:\/\/github.com\/unitycoder\/ffmpegStreamToUnity Instructions:– Download ffmpeg https:\/\/ffmpeg.org\/download.html– run from commandline: > […]<\/p>\n","protected":false},"author":1,"featured_media":4996,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2},"jetpack_post_was_ever_published":false},"categories":[660,804,3],"tags":[876,1099,1101,875,1100,1102,740],"class_list":["post-4995","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-opensource","category-scripts","category-unity3d","tag-bounty","tag-ffmpeg","tag-raw","tag-reward","tag-stream","tag-udp","tag-video"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/unitycoder.com\/blog\/wp-content\/uploads\/2019\/05\/ffmpeg-stream-to-unity.gif?fit=364%2C336&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p1KTaT-1iz","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/posts\/4995","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/comments?post=4995"}],"version-history":[{"count":3,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/posts\/4995\/revisions"}],"predecessor-version":[{"id":4999,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/posts\/4995\/revisions\/4999"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/media\/4996"}],"wp:attachment":[{"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/media?parent=4995"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/categories?post=4995"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/unitycoder.com\/blog\/wp-json\/wp\/v2\/tags?post=4995"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
![\"\"](\"https:\/\/i0.wp.com\/unitycoder.com\/blog\/wp-content\/uploads\/2019\/05\/ffmpeg-stream-to-unity.gif?resize=364%2C336&ssl=1\")