Monday, 18 November 2019

Week 3 [18-24.11.19] Write-A-Video

A global team of computer scientists, from Tsinghua and Beihang Universities in China, Harvard University in the US and IDC Herzliya in Israel, have developed "Write-A-Video", a new tool that generates videos from themed text. Using words and text editing, the tool automatically determines which scenes or shots are chosen from a repository to illustrate the desired storyline. The tool enables novice users to produce quality video montages in a simple and user-friendly manner that doesn't require professional video production and editing skills

1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?
2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?
3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?
4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

https://www.eurekalert.org/pub_releases/2019-11/afcm-csd111419.php

A video illustrating the project can be seen here:
https://vimeo.com/357657704

Full article:
http://miaowang.me/papers/a177-wang.pdf:

29 comments:

  1. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    I think that this idea could be very useful for professional development. Moreover it's awesome tools if we want to produce film based on book. Then this AI could prepare video which could be later using to generate professional movie (with additional effects).

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    For sure! That algorithm show that it is possible. Of course, this solution isn't perfect so now film editor are indispensable. To be honest, I think that in the future the film editors would be still necessary.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    I have (little) experience in video processing. I have a problem to answer...from one hand on the beginning that will be very usefull, but later when you have w vision and idea for movie, creating is peace of cake. Many tools are already so easy that you don't need many time to learn them.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    I read a few month ago about AI which are helpfull to "smoothing" cutted scenes (change light intensity and modify colours). The result was movie which contains many shorter scenes which have similar rapameters and when you watching them you wouldn't see that they have been made in other time of day.

    ReplyDelete
    Replies
    1. Thank you for your answer, I also think that AI will not replace film editors completely, but it will be able to help them a lot. The application you described in the last point sounds interesting, maybe it will look like this. The AI will not make the whole film, but will help with the more difficult elements of its creation.

      Delete
  2. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    Yes, I think it will be useful. This will help to make a demo version of a script and then it will be easier to evaluate. Also, I see this application in other areas, such as forensics

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    I think not, editors support - maybe, but not replace. Artificial intelligence is probably based on the work of editors with which it studied. Therefore, a person can see the editor in a completely different way, and perhaps better than artificial intelligence. Therefore, I do not believe that artificial intelligence will be able to replace this work, but it may well be an instrument of support

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?


    Yes, I have a little experience. I think that learning the basics will be easier, since it does not take much time.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    Yes, definitely. I had the opportunity to use a dipfake on deepfake=), I think this is a good enough tool. unless of course it is used correctly. For example, you could use him to select actors for new films


    ReplyDelete
    Replies
    1. Thanks for showing me an interesting application that didn't occur to me. In forensic applications it can indeed help to create showcase versions of events. I haven't tried using deepfake yet, but it seems so interesting that I will certainly try it one day.

      Delete
  3. This comment has been removed by the author.

    ReplyDelete
  4. At the first sight I though that it was just a simple content retrieval based on keywords, but in fact it's a really cool project! It may be very useful, I was working for a digital media company on projects helping video editors. One of the problems was assessing if a video will become popular or not and it was my focus. But another problem was creating those videos as the company had to produce hundreds of them on a daily basis. Experienced video editor was able to create ~3 videos a day. Such tool definitely would be helpful in that case, especially considering that the database wasn't a problem, the company had its own reporters and large repository of proprietary clips. Searching through that repository was the problem.

    The downside of this solution in the general context is that you need that repository of clips to select from. If it was about generating videos from text I think it could replace video editors in some cases but definitely not where you need well thought videos or maybe some more abstractive ones. After all a text mentioning giraffes doesn't have to be illustrated with a video showing them happily running through savanna.

    Personally I don't have experience in video processing related to creation of video montages. For individual purposes surely it would be easier to just learn how to create appealing montages by hand. In mass production cost of creation of such database and then generating videos almost automatically should pay off really soon.

    ReplyDelete
    Replies
    1. Than you for this answer. The problem of determining whether video will become popular sounds very interesting, especially when so many of them are produced.

      Delete
  5. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?
    Artificial intelligence also helps in the conversion and development of films. This allows you to create movies without outsourcing or any outside help with the help of appropriate video editors. In my opinion, artificial intelligence is the future of film editing in the case of
    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?
    In my opinion, artificial intelligence will certainly not replace professional editors. People without experience in film editing will benefit the most from it. Now the programs that use artificial intelligence allow it: Creating a compact storyline with pieces of film, Low cost and economy, Getting the right visual concept and style, Face detection and object tracking options, Automatic mute by reducing the volume of music or automatically creating sound effects around a dialog box, Creating organic-looking and realistic effects. To perform all these operations, you don't need to know how to do it.
    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?
    I don't have any experience editing movies. However, as far as the fundamentals are concerned, they will certainly be created if producers see that the use of artificial intelligence is commercially viable. Moreover, I believe that it is not so difficult if several scientists have done something like that, it will certainly be possible to apply it to a commercial project.
    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.
    I know the Magisto and Gliacloud. Both allow you to transform videos and photos into well made videos.

    ReplyDelete
    Replies
    1. Thank you for your reply and quoting Magisto and Gliacloud. I have never heard of them before and they seem very useful and easy to use. Maybe I will use them myself by creating some video from photos.

      Delete
  6. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    I will continue to put the man higher than the machine! Of course, such an idea has a chance to "flow out" but ... in most cases in the man versus machine combination as man versus food wins ... MAN! or WOMAN! ;-) of course.
    Seriously! A nice idea, I hope that we will be able to implement it and enable us to have, for example, an additional day off, which would be useful to us all!
    Cool idea!

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    As I wrote earlier, of course support yes, but whether to replace ... hmm .... I will, however, always support a man in a competition with a machine.
    You become monothematic Michał, but are we not going too much in AI, or the necessity to teach neural networks, when Albert Einstein has already described and specified many things? No comment ...

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    I'm sorry but I don't have such experiences. So it's difficult for me to say how video processing will be looks like in this case. I process images, not movies.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    Unfortunately I don't know ....

    ReplyDelete
    Replies
    1. Thank you for your answer and I'm also not so optimistic about the development of AI yet :)

      Delete
  7. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?
    I think it is a great idea. This tool is amazing and showing us the potential of AI. It could be used by many peoples which don’t have skills to prepare a good movie. I think in the future such tools using AI will be available in many fields of our live. Today we use AI in software available in smartphones to improve self made photos.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?
    I think in the future AI will suport editors but not replace them. I found a very interesting article about AI used in movies and entertainment. Link below:
    https://emerj.com/ai-sector-overviews/ai-in-movies-entertainment-visual-media/
    From this article you can read that AI helps editors in prepare film trailers.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?
    I have a little experience in processing videos. In the past I used Windows Movie Maker to prepare videos for my friends. I think that maybe it will be easier to learn basic in processing films than creating a videos database for AI. It’s all depends on people skills. We need a videos that describe our text that we wrote in application. It's not a easy task but for some people it will be the easiest way to prepare a good video. For some people learning basic in video processing is a hard task.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.
    In the past I used a Windows Movie Maker to prepare a simple video using pictures, sound and text. In short time we can prepare a good video and share them with friends. Another software that I was used was Avidemux. It's a tool that can cut scene in a simple way or combine two parts of film.

    ReplyDelete
    Replies
    1. Thank you for your response. I also remember I used to work in Windows Movie Maker, it was a very easy to use program but it allowed to take the first steps in video editing.

      Delete
  8. 1. I think this is a very interesting idea. Seeing the concept in the form of a video amazed me that such things are possible now. I think that this idea has a chance for wider application. This is a fun way for people not familiar with creating movies to create something simple and interesting, without having to learn complex programs. I would gladly use this idea myself and test how it works on various subjects. The only minus I see is a potentially small and thematically limited database at the moment.

    2. Certainly not replace. However, I think that suport is possible. After all, human creativity and experience, especially in film productions, outweigh the possibilities of AI at the moment. In my opinion, when editing films you need some craftsmanship and further discussions for the end result to satisfy viewers. It’s not just sticking selected fragments together.

    3. I have only had contact with video processing for a presentation once and I remember using Filmora as well. My editing consisted only of trimming another video in order to extract from it the fragments I needed. So it wasn't professional. Fortunately, working with this program was quite simple. In my opinion, creating a video database will definitely be difficult, because you need to collect a lot of videos on various topics. However, looking at the possibilities of this idea it’s profitable process. Of course, if you want to edit your own movies, learning the basics is still necessary.

    4. Unfortunately, I do not know any, but I will gladly follow the "Write-A-Video" project.

    ReplyDelete
    Replies
    1. Thank you. I am glad that I was able to find this project and I will also follow it.

      Delete
  9. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?
    I think that could be a helpful field for video editors working in the news agencies, where you have to prepare a lot of video material in minimal time.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?
    I think that would be a similar situation as with programmers (https://www.perforce.com/blog/qac/will-ai-replace-programmers). So, it could boost the process of video creation, rather than replace the human being.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?
    Probably yes, however, I could assume that some people could prefer to write a script than using the tool.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.
    I'm sorry, but it's not my field of interest. If you mean video processing by the meaning of "understanding the content," I'd go with OpenCV. However, if you mean the post-production like operations, I think that https://videohive.net/ database is very usefull!

    ReplyDelete
    Replies
    1. Thanks. You are another person who mentioned a large amount of production, maybe it will be a tool that will support large companies that produce huge amounts of films. The database you mentioned seems very extensive, I have to remember about it if I ever needed some movie inserts.

      Delete
  10. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?
    The idea seems a logical continuation of text comprehension systems. Once we can extract some meaning from the text, it's not that difficult to find some video clips matching extracted tags.
    I'm afraid this technique will allow for another torrent of dumb, algorithmically-generated junk videos targeting young children, like current animated ones (just search youtube for "finger song"). I don't think it'll be more than a curiosity in other applications.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?
    I think yes, it can support, but not like that. It will allow for abstracting the whole process a bit, like editor won't be matching the color profiles of two scenes by hand but will just click "I want to have it matched" and that's it. I don't think it's possible to replace - editing is more than technical process, there is whole language of picture that is intractable to specify in a formal way.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?
    I know some basics.
    The thing is that such database has to be created once, and then can be reused. It can be created in semi- or fully-automatic way. (The biggest problem is clearing licences). It's a bit like with labor and capital gains.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.
    The whole topic of style transfer neural networks I think would apply. But I'm not well oriented in those things.

    ReplyDelete
    Replies
    1. Thanks for the answer. I have nothing to add, I agree with everything you've written.

      Delete
  11. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    It is an interesting idea. In my opinion such tools may be used as a support for people with reading disabilities. people who have had a brain disease (e.g. stroke), or after brain injuries, often suffer from negative conseqences after, like difficulties with reading. This kind of tool might help to understand the text for those who does not understand written content. Maybe the application culd also be used to help such people to learn to read again.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    Maybe in simple tasks, but generally I do not believe that Artificial Intelligence is able to replace people in a position that requires creativity, like creating movies. This type of work relates to the feelings and emotions of viewers.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    I do not have experience in video processing, but after reading an article and watching a video, creating short movies with the presented tool seems to be easy.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    Actually I am not much interested in video processing, so I am not up to date and don't know latest news.

    ReplyDelete
    Replies
    1. Thanks for mentioning another possible application for this technology.

      Delete
  12. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    It's hard to say. Especially because it doesn't remove the creative input, just moving to a different domain (text). And there is a large corpus of text already available - so it can be just a way of enhancing the delivery of textual works.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    Support yes, eliminate - I don't think so. I like watching unedited vlogs, so for me there could be no video editors right now.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    No, I have almost none. Thus, I can't weigh in on this subject. It seems that "everybody can make a described video database", but it can get difficult at scale. So maybe there will be video databases for purchase.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    Not from the AI realm. The farthest I've been is using batch processing with FFMPEG to adjust video quality, size and duration.

    ReplyDelete
    Replies
    1. Thanks for the answer. I agree that such databases could be created by large companies that could later sell access to them.

      Delete
  13. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    I think word video processing is a very nice idea that can help anyone interested in creating amateur video. Professionals will rather stick to the "old methods" for the time being. However, this method of creating films will surely gain many followers and begin to develop

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    Artificial intelligence already supports editors on many issues, for example, graphics (photos) processing. I think that with time it will go everywhere, whenever possible, also as film editors. It is important that the last word always belongs to the person who is responsible for the whole.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    Earlier I was interested in image processing and I know that artificial intelligence is able to do great things, for example, it can choose layers by itself.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    I think that a good example may be Google Photos, which can, for example, recognize the person we took a photo of and suggest to share it or recognize a photo of a document and can export it to a pdf file.

    ReplyDelete
    Replies
    1. Thanks. Looking at it from this side, all that can be used for image processing can also be used for video. And as you know, AI is widely used when it comes to photos.

      Delete
  14. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    It is really interesting topic! I have never heard about tool which enables writing the text and then “converts” it on video. The video which illustrates how tool can be used is very interesting. I think that this idea can be applied widely. In my opinion the mentioned idea can be used in learning for example foreign language. Nowadays, teachers try to prepare materials with some multimedia and the good way to learn language is watching movies or listening songs. It is important that the selected movie should be related to the topic which is currently discussed and it is sometimes very difficult to find some suitable movie because of topic of movie and with level of language as well. Teacher can write the description and generate the movie so it means that the movie will contain what the teacher wants no more.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?

    In my opinion Artificial Intelligence will be able to support film editors in the future or even replace them. AI is very fast developing technology and the world science is focused on AI but I suppose that AI technology is not developed to the degree allowing to support film editors now. I guess that nowadays, there are few cases when AI is applied in editing films but they are starting appears. However, I think that it will be more popular in next 10 years. You asked about replacing film editors by AI – I suppose that it will be possible but not in our generation. I think that the situation of replacing human being will be very often the case of most jobs in the future and not only the job of film editor.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    I have a little experience in video processing because I create videos from my trips. When I come back home, I try to create one video including the most interesting and funny activities from my trips. I have a lot of short videos and I want to join then into one video, cut off some parts of video, slow down or speed up video. Often I have problems that some parts are falling apart and I am angry that something is going wrong but I like it. It means that I do not choose option with creating the described video database instead of manually creating movie. However, I think that some of people can say that creating the described video database is more useful than learning the basics.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.

    I did not know other projects that make work with video processing easier, because I am not interested in video processing. I create video only several times per year to show my friend how my trip looked like. I found some interesting article in the Internet: https://analyticsindiamag.com/top-5-ai-powered-video-editing-tools/ It is 5 Top AI- Powered video editing tools.

    ReplyDelete
    Replies
    1. Thank you for your answer. You mentioned another interesting application, in fact, linking texts with film can make it easier to remember the language. Thanks also for the link, it's great that already so many tools are supported by AI.

      Delete
  15. 1. What do you think about this idea, does it have a chance to be used more widely or is it just a curiosity?

    The solution is available rather as a curiosity, although the larger the database with movies is, the better this tools are. I wonder about the copyright of such films.

    2. Do you think Artificial Intelligence will be able to support or replace film editors in the future?
    I think they can help with doing repetitive work. However, for sure in the near future a man in creative work will not be replaced.

    3. Do you have experience in video processing? Don't you think that learning the basics will be easier than creating the described video database?

    I have minimal experience in editing - but cutting out several films and assembling them into one is really easy.

    4. Are you familiar with other projects that make it easier to work with video processing? If so, tell us about it.
    In video processing I am delighted with the digital image stabilization - I must say that it works better and better. An example would be the Warp Stabilizer effect in Adobe CC.

    ReplyDelete
    Replies
    1. Thanks for a specific example of current AI support.

      Delete